METHOD OF AND UNIT FOR INPUTTING AN IMAGE, AND 
COMPUTER PRODUCT 

FIELD OF THE INVENTION 
5 The present invention relates to an unit for inputting 

an image ("image input unit") that can perform pixel shift 
photography to obtain a high quality image . Moreover, this 
invention relates to a method of inputting an image ("image 
input method"), and a computer-readable recording medium 
10 that stores a computer program for realizing the method. 

BACKGROUND OF THE INVENTION 

With the development of solid-state image sensing 
devices, image input equipment such as small video cameras 

15 for household use, digital cameras, for personal computers, 
etc., have rapidly become widespread. In the case of the 
video camera, sufficient image quality can be obtained from 
a standpoint of television standard such as NTSC. CCD is 
normally used as the image sensing device. However, the 

20 CCD reads two fields twice as one frame corresponding to 
the NTSC and PAL methods, and the majority thereof has an 
oblong pixel. 

However, recently, the object is to display images 
picked-up using a digital still camera on a display of a 

25 personal computer or the like . As a result , a CCD that reads 



all the square pixels (i.e. the progressive scan type) have 
been used as the image sensing device. 

Thus, now a days the images picked-up with the digital 
still camera are mostly edited using a personal computer 
and printed using a printer . However, if a large-sized image 
is to be obtained, or a resolution that is necessary as a 
hard copy or computer graphics or the like is to be obtained, 
then the number of pixels provided in the present day digital 
still camera are not sufficient. A solid-state image 
sensing device having a higher number of pixels becomes 
necessary in these purposes. However, a screen size of the 
solid-state image sensing device becomes larger if the 
resolution is increased. As the screen size becomes larger, 
the lens section has to be made larger so that a down-sizing 
of the digital still camera becomes difficult. Moreover, 
a cost of such the solid-state image sensing device increases 
and can it not be used in the low-cost widespread digital 
cameras . 

As a method of realizing high quality by the solid-state 
image sensing device, a so-called method of "pixel shift 
photography" has been well known. The pixel shift 
photography is a method of obtaining a high resolution by 
combining an image photographed by shifting a subject by 
half of a pixel pitch and an image before such shifting to 
obtain one image. Typical examples of the conventional 



image shift mechanism are shown in Fig. 27A to Fig. 27C. 
Fig. 27A shows a CCD shift method. In this CCD shift method, 
the image sensing device is shifted to obtain images that 
are different from each other. Fig. 27B shows a LPF gradient 
method. In this LPF gradient method, a LPF is inclined to 
obtain images that are different from each other. Fig. 27C 
shows a lens shift method. In this lens shift method, a 
lens is shifted to obtain images that are different from 
each other. 

A desired image quality may not be obtained in the 
pixel shift photography because of the events such as a 
movement of hands, a movement of the subject, or a change 
in a performance of the pixel shift mechanism. In each of 
these events, the degree of deterioration of the image 
differs depending on the degree thereof. If the 
deterioration of the image crosses a certain level, the image 
may become inferior as compared to an image obtained in 
ordinary photography. The ordinary photography is the one 
that does not apply the technique of the pixel shift 
photography . 

The cause of the change in the performance of the pixel 
shift mechanism may as follows . That is, an amount of shift 
of the pixel shift mechanism may change slightly due to a 
lapse of time, or the amount of shift of the pixel shift 
mechanism may change temporarily due to a change in the 



atmospheric temperature, or the pixel shift mechanism may 
fail due to vibration or impact and whereby the normal 
operation cannot be performed. The pixel shift mechanism 
requires micro-level displacement at high accuracy in 
comparison to any other mechanical sections of the digital 
camera. Therefore, the pixel shift mechanism is likely to 
be affected readily by the lapse of time, change in the 
temperature, and the influence of vibrations and impact. 
Thus, it is very likely that the pixel shift mechanism 
becomes defective permanently or temporarily. 

For example, there is a case where a chemical reaction 
of curing progresses slightly even in a cured state of an 
adhesive, which would be judged as "being sufficiently cured" 
for normal use, and this slight difference will largely 
affect the pixel shift. Moreover, a slight change in 
rigidity of a structure due to temperature, a slight change 
in a coefficient of friction due to a change with the lapse 
of time, and a minute shift of a contact point of a structure 
due to an impact may cause a serious problem in the pixel 
shift, though these may be considered only as an error in 
a normal mechanism. 

Moreover, if each of the movement of hands , the movement 
of the subject, or the change in the performance of the pixel 
shift mechanism is slight, although a higher quality can 
be obtained, the image quality is not near to the desired 



one . Deterioration of the quality of the images hereinmeans 
a deterioration of resolution, an increase of blurred color, 
or the like. 

Fig. 28A to Fig. 28C are diagrams for explaining the 
principle of pixel shift and image deterioration. A CCD 
having the Bayor array is taken as an example. As shown 
in Fig. 28A, in normal photographing, R, G and B signals 
of the CCD are interpolated to obtain R, G and B signals 
for each pixel . Fig. 28B shows an ideal state when the pixel 
shift photography (a shift by one pixel in the longitudinal 
direction) is performed. In the case of this pixel shift 
photography, an image having a high resolution can be 
obtained, and interpolation is not required. On the other 
hand, Fig. 28C shows a state when an accuracy of the pixel 
shift photography (a shift by one pixel in the longitudinal 
direction) is insufficient. In the case shown in Fig. 28C, 
the resolution of the image is deteriorated, and blurred 
color occurs. 

A further problem is that a type of a LCD monitor 
normally mounted on a digital camera has a small number of 
displayed pixels. Therefore, even if there is a 
deterioration in the image quality, it can not be detected. 
In most cases, the deterioration in the image quality is 
recognized only after the image is transferred to a personal 
computer. Thus, failure of the pixel shift photography 



cannot be judged only with a digital camera (i.e., for example 
the personal computer is required) . 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide 
an image input unit that can prevent unsuccessful 
photographing in the pixel shift photography as much as 
possible;, and with which a photographer can obtain a desired 
high quality image, even if movement of hands, movement of 
the subject, or a performance change of the pixel shift 
mechanism occurs. It is another object of this invention 
to provide an image input method and a computer-readable 
recording medium that stores a computer program for realizing 
the method according to the present invention. 

The image input unit according to one aspect of this 
invention is capable of performing pixel shift photography. 
This image input unit comprises: a photographic optical 
system which forms an image of a subject in a predetermined 
position; an image sensing unit which generates image data 
corresponding to the image of the subject; a pixel shift 
mechanism which displaces the image sensing unit by a 
predetermined amount; a pixel shift mechanism control unit 
which controls the pixel shift mechanism so as to displace 
the image sensing unit by the predetermined amount; an image 
combining unit which generates image data for one image by 



combining the image data for a plurality of images output 
before and after the displacement of the image sensing unit; 
and a judgment unit which judges whether the pixel shift 
photography has been normally performed or not, based on 
the image data for a plurality of images output before and 
after the displacement of the image sensing unit. 

According to the above invention, the photographic 
optical system forms an image of the subject in the 
predetermined position. The image sensing unit generates 
image data corresponding to the image of the subject. The 
pixel shift mechanism control unit controls the pixel shift 
mechanism to displace the image sensing unit by a 
predetermined amount. The image combining unit generates 
image data for one image by combining image data for the 
plurality of images output before and after the displacement 
of the image sensing unit. The judgment unit judges whether 
the pixel shift photography has been normally performed or 
not, based on the image data for a plurality of images output 
before and after the displacement of the image sensing unit. 

The image input unit according to another aspect of 
this invention is capable of performing pixel shift 
photography. This image input unit comprises: a 
photographic optical system which forms an image of a subject 
in a predetermined position; an image sensing unit which 
generates image data corresponding to the image of the 



subject; a pixel shift mechanism which displaces the image 
sensing unit by a predetermined amount; a pixel shift 
mechanism control unit which controls the pixel shift 
mechanism so as to displace the image sensing unit by the 
predetermined amojunt; an image combining unit which 
generates image data for one image by combining the image 
data for a plurality of images output before and after the 
displacement of the image sensing unit; a pixel shift 
evaluation value calculation unit which calculates a pixel 
shift evaluation value for judging whether the pixel shift 
photography has been normally performed or not, based on 
-the image data for a plurality of images output before and 
after the displacement of the image sensing unit; a storage 
unitwhich stores the pixel shift evaluation value calculated 
by the pixel shift evaluation value calculation unit; a 
storage control which controls storage of the pixel shift 
evaluation value in the storage unit; and a failure judgment 
unit which judges a failure of the pixel shift mechanism, 
based on the pixel shift evaluation values obtained by 
performing the pixel shift photography for several times 
and stored in the storage unit. 

According to the above invention, the photographic 
optical system forms an image of a subject in a predetermined 
position, the image sensing unit generates image data 
corresponding to the image of the subject, the pixel shift 



mechanism control unit controls the pixel shift mechanism 
to displace the image sensing unit by a predetermined amount . 
The image combining unit generates image data for one image 
by combining image data for the plurality of images output 
before and after the displacement of the image sensing unit. 
The pixel shift evaluation value calculation unit calculates 
a pixel shift evaluation value for judging whether the pixel 
shift photography has been normally performed or not, based 
on the image data for a plurality of images output before 
and after the displacement of the image sensing unit. The 
storage control unit stores the pixel shift evaluation value 
calculated by the pixel shift evaluation value calculation 
unit in the storage unit, and the failure judgment unit judges 
a failure of the pixel shift mechanism, based on the pixel 
shift evaluation values obtained by performing the pixel 
shift photography for several times and stored in the storage 
unit. 

The image input unit according to still another aspect 
of this invention is capable of performing pixel shift 
photography. This image input unit comprises: a 
photographic optical system which forms an image of a subject 
in a predetermined position; an image sensing unit which 
generates image data corresponding to the image of the 
subject; a pixel shift mechanism which displaces the image 
sensing unit by a predetermined amount; an adjustment value 



storage unit which stores an adjustment value for driving 
the pixel shift mechanism; a pixel shift mechanism control 
unit which controls the pixel shift mechanism based on the 
adjustment value stored in the adjustment value storage unit, 
to thereby displace the image sensing unit by a predetermined 
amount; an image combining unit which generates image data 
for one image by combining the image data for a plurality 
of images output- before and after the displacement of the 
image sensing unit; a pixel shift evaluation value 
calculation unit which calculates a pixel shift evaluation 
value for judging whether the pixel shift photography has 
been normally performed or not, based on the image data for 
a plurality of images output before and after the 
displacement of the image sensing unit; a selection unit 
which selects the pixel shift calibration mode; and a self 
calibration unit which executes pre-pixel shift photography 
once or plural times, when the pixel shift calibration mode 
is selected, and re-calculates the adjustment value based 
on the pixel shift evaluation value for each pixel shift 
photography, calculated by the pixel shift evaluation value 
calculation unit, to thereby calculate a new ad j ustment value , 
and changes the adjustment value stored in the adjustment 
value storage unit to the new adjustment value. 

According to the above invention, the photographic 
optical system forms an image of the subject in the 



predetermined position, the image sensing unit generates 
image data corresponding to the image of the subject, and 
the pixel shift mechanism control unit controls the pixel 
shift mechanism based on the adjustment value stored in the 
adjustment value storage unit to displace the image sensing 
unit by a predetermined amount. The image combining unit 
generates image data for one image by combining image data 
for the plurality of images output before and after the 
displacement of the image sensing unit. The pixel shift 
evaluation value calculation unit judges whether the pixel 
shift photography has been normally performed or not, based 
on the image data for a plurality of images output before 
and after the displacement of the image sensing unit. The 
selection unit selects the pixel shift calibration mode. 
When the pixel shift calibration mode is selected, the self 
calibration unit executes pre-pixel shift photography once 
or plural times, re-calculates the adjustment value based 
on the pixel shift evaluation value for each pixel shift 
photography, calculated by the pixel shift evaluation value 
calculation unit, to thereby calculate a new ad j us tment value , 
and changes the adjustment value stored in the adjustment 
value storage unit to the new adjustment value. 

The image input unit according to still another aspect 
of this invention is capable of performing pixel shift 
photography. This image input unit comprises: a 



photographic optical system which forms an image of a subject 
in a predetermined position; an image sensing unit which 
generates image data corresponding to the image of the 
subject; a pixel shift mechanism which displaces the image 
sensing unit by a predetermined amount; an adjustment value 
storage unit which stores an adjustment value for driving 
the pixel shift mechanism; a pixel shift mechanism control 
unit which controls the pixel shift mechanism based on the 
adjustment value storedin the adj us tment value storage unit, 
to thereby displace the image sensing unit by a predetermined 
amount; an image combining unit which generates image data 
for one image by combining the image data for a plurality 
of images output before and after the displacement of the 
image sensing unit; a pixel shift evaluation value 
calculation unit which calculates a pixel shift evaluation 
value for judging whether the pixel shift photography has 
been normally performed or not, based on the image data for 
a plurality of images output before and after the 
displacement of the image sensing unit; and a pixel shift 
photography control unit which controls to execute an 
operation repetitively in which, when a continuous pixel 
shift mode is selected, a pre-pixel shift photography is 
executed by setting an adjustment value in the pixel shift 
mechanism, the pixel shift evaluation value calculation unit 
calculates the pixel shift evaluation value to thereby 



calculate a new adjustment value based on the calculated 
pixel shift evaluation value, and pre-pixel shift 
photography is performed again with the calculated new 
adjustment value. 

According to the above invention, the photographic 
optical system forms an image of the subject in the 
predetermined position, the image sensing unit generates 
image data corresponding to the image of the subject, and 
the pixel shift mechanism control unit controls the pixel 
shift mechanism based on the adjustment value stored in the 
adjustment value storage unit to displace the image sensing 
unit by a predetermined amount. The image combining unit 
generates image data for one image by combining image data 
for the plurality of images output before and after the 
displacement of the image sensing unit. The pixel shift 
evaluation value calculation unit calculates a pixel shift 
evaluation value for judging whether the pixel shift 
photography has been normally performed or not, based on 
the image data for a plurality of images output before and 
after the displacement of the image sensing unit. The pixel 
shift photography control unit controls to execute an 
operation repetitively in which, when a continuous pixel 
shift mode is selected, a pre-pixel shift photography is 
executed by setting an adjustment value in the pixel shift 
mechanism, the pixel shift evaluation value calculation unit 



calculates the pixel shift evaluation value to thereby 
calculate a new adjustment value based on the calculated 
pixel shift evaluation value, and pre-pixel shift 
photography is performed again with the calculated new 
adjustment value. 

The image input method, capable of performing pixel 
shift photography, according to still another aspect of this 
invention comprises the steps of: acquiring a first image 
data corresponding to an image of a subject with an image 
sensing unit; displacing the image sensing unit by a 
predetermined amount; acquiring a second image data 
corresponding to an image of the subject with the image 
sensing unit after the displacement of the image sensing 
unit; generating image data for one image by combing the 
first image data and the second image data; and judging 
whether the pixel shift photography has been normally 
performed or not, based on the first image data and the second 
image data. 

According to the above invention, a subject image is 
imaged on the image sensing unit to output first image data, 
the image sensing unit is displaced by a predetermined amount , 
the subject image is imaged on the image sensing unit 
displaced by the predetermined amount to output second image 
data, image data for one image is generated based on the 
first image data and the second image data, and it is judged 



whether the pixel shift photography has been normally 
performed or not, based on the first image data and the second 
image . 

The image input method, capable of performing pixel 
shift photography, according to still another aspect of this 
invention comprises the steps of: acquiring a first image 
data corresponding to an image of a subject with an image 
sensing unit; displacing the image sensing unit by a 
predetermined amount by a pixel shift mechanism; acquiring 
a second image data corresponding to an image of the subject 
with the image sensing unit after the displacement of the 
image sensing unit; generating image data for one image by 
combining the first image data and the second image data; 
calculating a pixel shift evaluation value for judging 
whether the pixel shift photography has been normally 
performed or not, based on the first image data and the second 
image data; storing the pixel shift evaluation value in a 
storage unit; and judging a failure or normal operation of 
.the pixel shift mechanism, based on the pixel shift 
evaluation values obtained by performing the pixel shift 
photography for several times and stored in said storage 
unit and are stored in the storage unit. 

According to the above invention, a subject image is 
imaged on the image sensing unit to output first image data, 
the image sensing unit is displaced by a predetermined amount 



by the pixel shift mechanism, the subject image is imaged 
on the image sensing unit displaced by the predetermined 
amount to output second image data, and image data for one 
image is generated based on the first image data and the 
second image data. The pixel shift evaluation value for 
judging whether thepixel shift photography has beennormally 
performed or not is calculated based on the first image data 
and the second image data . The pixel shift evaluation value 
is stored in the storage unit, and it is judged if there 
is a failure of the pixel shift mechanism, based on the pixel 
shift evaluation values obtained by performing the pixel 
shift photography for several times and stored in said 
storage unit and stored in the storage unit. 

The image input method, capable of performing pixel 
shift photography, according to still another aspect of this 
invention comprises the steps of: acquiring a first image 
data corresponding to an image of a subject with an image 
sensing unit; displacing the image sensing unit by a 
predeterminedamount by driving a pixel shift raechanismbased 
on an adjustment value stored in a storage unit; acquiring 
a second image data corresponding to an image of the subject 
with the image sensing unit after the displacement of the 
image sensing unit; generating image data for one image based 
on the first image data and the second image data; and 
executing pre-pixel shift photography once or plural times. 



when a self calibration mode is selected to calculate a pixel 
shift evaluation value for judging whether the pixel shift 
photography has been normally performed or not based on the 
first image data and the second image data; and calculating 
a new adj ustment value by re -calculating the ad j ustment value , 
based on the calculated pixel shift evaluation value; to 
change the adjustment value stored in the storage unit to 
the new adjustment value. 

According to the above invention, a subject image is 
imaged on the image sensing unit to output first image data, 
the pixel shift mechanism is driven based on an adjustment 
value stored in a storage unit to displace the image sensing 
unit by a predetermined amount, and the subject image is 
imaged on the image sensing unit displaced by the 
predetermined amount to output second image data . One image 
data is generated based on the first image data and the second 
image data, and when the self calibration mode is selected, 
pre-pixel shift photography is executedonce or plural times, 
to calculate a pixel shift evaluation value for judging 
whether the pixel shift photography has been normally 
performed or not based on the first image data and the second 
image data . Based on the calculated pixel shift evaluation 
value, anew adjustment value is calculated by re-calculating 
the adjustment value, to change the adjustment value stored 
in the storage unit to the new adjustment value. 



The image input method, capable of performing pixel 
shift photography, according to still another aspect of this 
invention comprises: a first step of acquiring a first image 
data corresponding to an image of a subject with an image 
sensing unit; a second step of displacing the image sensing 
unit by a predetermined amount by driving a pixel shift 
mechanism based on an adjustment value stored in a storage 
unit; a third step of acquiring a second image data 
corresponding to an image of the subject with the image 
sensing unit after the displacement of the image sensing 
unit; a fourth step of calculating a pixel shift evaluation 
value for judging whether the pixel shift photography has 
been normally performed or not based on the first image data 
and the second image data; a fifth step of calculating a 
new adjustment value by re-calculating the adjustment value, 
based on the calculated pixel shift evaluation value, and 
changing the adjustment value stored in the storage unit 
to the new adjustment value; a sixth step of repeating the 
first to fifth steps for a predetermined number of times, 
or until the new adjustment value becomes within a 
predetermined range; and a seventh step of performing pixel 
shift photography with the final adjustment value. 

According to the above invention, a subject image is 
imaged on the image sensing unit to output first image data, 
the pixel shift mechanism is driven based on an adjustment 



value stored in a storage unit to displace the image sensing 
unit by a predetermined amount, and the subject image is 
imaged on the image sensing unit displaced by the 
predetermined amount to output second image data . The pixel 
5 shift evaluation value for judging whether the pixel shift 
photography has been normally performed or not is calculated 
based on the first image data and the second image data. 
A new adjustment value is calculated by re-calculating the 
adjustment value based on the calculated pixel shift 

10 evaluation value, to change the adjustment value stored in 
the storage unit to the new adjustment value. The above 
operation is repeated for a predetermined number of times, 
or until the new adjustment value becomes within a 
predetermined range, and pixel shift photography is 

15 performed with the final adjustment value. 

The computer readable recording medium according to 
another aspect of the present invention stores a computer 
program which when executed realizes the method according 
to the present invention. 

20 Other objects and features of this invention will 

become apparent from the following description with 
reference to the accompanying drawings. 



BRIEF DESCRIPTION OF THE DRAWINGS 
25 Fig. 1 is a block diagram showing the construction 

1 9 



of a digital camera according to a first embodiment; 

Fig. 2 is a flowchart for explaining the operation 
of pixel shift photography of a digital camera according 
to the first embodiment; 
5 Fig. 3 is a diagram showing a display example of a 

camera display section in Fig. 1; 

Fig. 4 is a diagram showing a display example of the 
camera display section in Fig. 1; 

Fig . 5A to Fig . 5C are diagrams for explaining a method 
10 of calculating the coincidence degree of an image in a pixel 
shift evaluation value and reliability calculation section 
in Fig. 1; 

Fig. 6A and Fig. 6B are diagrams for explaining a method 
of calculating the amount of shift of an image in the pixel 
15 shift evaluation value and reliability calculation section; 

Fig. 7A to Fig. 7D are diagrams for explaining a method 
of calculating the amount of shift of an image in the pixel 
shift evaluation value and reliability calculation section; 
Fig . 8A to Fig . 8D are diagrams for explaining a method 
20 of calculating the amount of shift of an image in the pixel 
shift evaluation value and reliability calculation section; 

Fig. 9AtoFig. 9D are diagrams for explaining a method 
of calculating the amount of shift of an image in the pixel 
shift evaluation value and reliability calculation section; 
25 Fig. lOA to Fig. lOD are diagrams for explaining a 



method of calculating the amount of shift of an image in 
the pixel shift evaluation value and reliability calculation 
section; 

Fig. 11 is a diagram for explaining a method of 
calculating the amount of shift of an image in the pixel 
shift evaluation value and reliability calculation section; 

Fig . 12 is a diagram showing one example of an operation 
area in an image frame; 

Fig. 13A and Fig. 13B are diagrams for explaining the 
relation between the imaged image before and after a shift 
of an image sensing unit and the amount of shift in each 
operation area; 

Fig. 14A and Fig. 14B are diagrams for explaining the 
relation between the imaged image before and after a shift 
of an image sensing unit and the amount of shift in each 
operation area; 

Fig. 15A and Fig. 15B are diagrams for explaining the 
relation between the imaged image before and after a shift 
of an image sensing unit and the amount of shift in each 
operation area; 

Fig. 16A and Fig. 16B are diagrams for explaining the 
relation between the imaged image before and after a shift 
of an image sensing unit and the amount of shift in each 
operation area; 

Fig. 17 is a flowchart for explaining the operation 



of pixel shift photography of a digital camera according 
to a second embodiment; 

Fig. 18 is a diagram showing a display example of the 
camera display section; 
5 Fig. 19 is a flowchart for explaining the operation 

of pixel shift photography of a digital camera according 
to a third embodiment; 

Fig. 20A and Fig. 20B are diagrams for explaining a 
method of calculating an adjustment value in the self 
10 calibration mode; 

Fig. 21A to Fig. 21H are diagrams for explaining the 
method of calculating an adjustment value in the self 
calibration mode; 

Fig. 22 is a diagram for explaining the method of 
15 calculating of an adjustment value in the self calibration 
mode ; 

Fig. 23 is a diagram for explaining a method of 
calculating an optimum adjustment value from adjustment 
values in two points; 
20 Fig. 24 is a diagram for explaining a method of 

calculating an optimum adjustment value from adjustment 
values in two points; 

Fig. 25 is a diagram for explaining a method of 
calculating an optimum adjustment value from an adjustment 
25 values in one point; 



Fig. 26 is a flowchart for explaining the operation 

of pixel shift photography of a digital camera according 

to the second embodiments- 
Fig. 27A to Fig. 27C are diagrams for explaining a 
5 representative example of a conventional pixel shift 

mechanism; and 

Fig. 28A to Fig. 28C are diagrams for explaining the 

principle of pixel shift and image deterioration. 

10 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Embodiments of a digital camera applying the image 
input unit and the image input method according to the present 
invention will now be described in detail below with 
reference to the accompanying drawings. In this 

15 specification, the term "pixel shift evaluation value" is 
used to mean both "an amount of shift" and "a degree of 
coincidence" ("coincidence degree"). 

The digital camera according to the first embodiment 
will now be described with reference to Fig. 1 to Fig. 16B. 

20 Fig. 1 is a block diagram showing the construction of a digital 
camera according to the first embodiment. 

Fig. 1 is a block diagram showing the construction 
of the digital camera 1 according to this embodiment. This 
digital camera 1 comprises a body tube unit 2 for obtaining 

25 the analog image data by imaging a subject, and a signal 



processing unit 3 for processing digital image data obtained 
by digitalizing the analog image data obtained by this body 
tube ^unit 2. The signal processing unit 3 outputs the 
processed data to the outside. 
5 The digital camera 1 is provided with an A/D converter 

4 for A/D converting the image data is provided between the 
body tube unit 2 and the signal processing unit 3. A camera 
operation section 8 is provided for generating a signal 
depending on the operation of a mode switch (not shown) or 

10 the like and outputting this signal to the signal processing 
unit 3 . A camera display section 7 is provided for displaying 
the photographed image data or the like . This camera display 
section 7 is connected to the signal processing unit 3. A 
pixel shift drive circuit 6 drives a laminated piezoelectric 

15 element 26 of a pixel shift unit 20 in accordance with an 
adjustment value output from a system controller 30 . A body 
tube drive circuit 5 drives a lens drive system 14 in 
accordance with a control signal output from the system 
controller 30. 

20 The body tube unit 2 is constituted of an optical unit 

10 and the pixel shift unit 20 . The optical unit 10 comprises 
a lens 11 for forming an image ofasubject, a shutter mechanism 
12 (mechanical shutter, stop, etc.) for shading the light 
shone onto the lens 11 with an optical mechanism, an LPF 

25 (low-pass filter) 13, and a lens drive system 14 for moving 



the lens 11 and driving the shutter mechanism 12, in 
accordance with the control of the signal processing unit 
3 . 

The pixel shift unit 20 has an image sensing device 
unit 25 including an image sensing device for receiving the 
light shone onto the lens 11 and converting the light to 
an electric signal (analog image data) , and a laminated 
piezoelectric element 26 arranged on the back side of the 
image sensing device for performing a pixel shift by moving 
the light-receiving plane of the image sensing device in 
parallel to the image-forming surface (in the vertical 
direction with respect to the optical axis) , in accordance 
with the control of the signal processing unit 3, to change 
the imaging range of the same subject. The output of the 
image sensing device is supplied to the A/D converter 4 . 

The signal processing unit 3 has, for example, the 
system controller 30, a data output section 31, a memory 
group 32 consisting of a plurality of memories, an image 
processing section 33, and a pixel shift evaluation value 
and reliability calculating section 34. 

The data output section 31 is connected to the output 
of the A/D converter 4 and inputs the digital image data, 
and outputs the input digital image data to either one of 
memories in the memory group 32 on the subsequent stage, 
in accordance with the control of the system controller 30. 



The memory group 32 has a plurality of memories/ and 
uses one memory for storing image data for one imaging, and 
reads the digital image data from one memory to supply the 
digital image data to the image processing section 33 on 
5 the subsequent stage, under control of the system controller 
30 . 

The pixel shift evaluation value and reliability 
calculating section 34 calculates a pixel shift evaluation 
value for judging whether the pixel shift photography has 

10 been normally performed or not based on the image data before 
and after the pixel shift stored in the memory group 32, 
and reliability data indicating the reliability of this pixel 
shift evaluation value, and outputs these data to the system 
controller 30. As described above, the pixel shift 

15 evaluation value and the reliability data are calculated 
for each of a plurality of operation areas in the image frame 
(see Fig. 12) . As for the reliability data, for example, 
a value evaluating the contrast of the image data is used. 
The image processing section 33 is for performing image 

20 combiningprocessingbased on the digital image data supplied 
from the memory group 32. Specifically, the image 
processing section 33 obtains a high resolution picture in 
the same subject corresponding to the amount of shift of 
pixel (for example, shifting one pixel in the longitudinal 

25 direction) , at the time of changing the imaging range by 



the laminated piezoelectric element 26, and finally image 
data for one sheet is obtained. 

The output of this image processing section 33 is 
connected to an external terminal (not shown) , and by 
connecting it to an external equipment such as a personal 
computer, the data can be transmitted . The camera operation 
section 8 includes a release key for operating an electronic 
shutter or a shutter mechanism 12 to perform imaging, a mode 
button for setting various modes (pixel shift photography 
mode) or the like. 

The camera display section 7 consists of, for example, 
an LCD, and image data and necessary information are 
displayed thereon, in accordance with the display control 
of the system controller 30. The system controller 30 
displays the result of judgment whether the pixel shift 
photography has been normally performed or not in the camera 
display section 7 for informing the user, when the user 
performs the pixel shift photography. In the case where 
the abnormal situation in the pixel shift photography is 
informed, not only the display on the display screen but 
also voice, alarm sound, vibrations or the like may be used 
for informing the user of the abnormality. 

The system controller 30 is connected to each of the 
lens drive system 14 of the optical unit 10, the image sensing 
device and the laminated piezoelectric element 26 of the 



pixel shift unit 20, the A/D converter 4, the data output 
sectionSl, the memory group 32 , the image proces sing section 
33, the body tube drive circuit 5, the pixel shift drive 
circuits, the camera display section 7 , the camera operation 
section 8, and the pixel shift drive circuit 6, for 
controlling the imaging operation, the A/D conversion, read 
and write of the memory, pixel shift and operations in 
response to the key input. The system controller 30 is 
constituted of a microcomputer or the like, and operates 
the microcomputer in accordance with various programs stored 
in the ROM beforehand to thereby execute control of each 
unit and the operation processing. In the pixel shift 
photography mode, the system controller 30 judges whether 
the pixel shift photography has been normally performed or 
not, based on the pixel shift evaluation value and the 
reliability data calculated by the pixel shift evaluation 
value and reliability calculating section 34, and displays 
the judgment result in the camera display section 7. 

The pixel shift photography operation of the digital 
camera in Fig. 1 will now be described with reference to 
the flowchart in Fig. 2. When an instruction to start the 
pixel shift photography is input from the camera operation 
section 8 (step SI), the first photographing is performed 
and the image data is stored in the memory group 32 (step 
S2) . Then, the system controller 30 outputs an adjustment 



value for shifting the image sensing device by one pixel 
in the longitudinal direction to the pixel shift drive 
circuit 6, to thereby drive the pixel shift unit 20 via the 
pixel shift drive circuit 6, so that the image sensing device 
5 is shifted (pixel shift) by one pixel in the longitudinal 
direction (step S3). The second photographing is per formed 
with the image sensing device shifted by one pixel in the 
longitudinal direction, and the image data is stored in the 
memory group 32 (step S4) . 

10 The image processing section 33 generates the image 

data having a high resolution, by combining the first image 
data and the second image data stored in the memory group 
32 (step S5). After completion of the pixel shift 
photography, the pixel shift evaluation value and 

15 reliability calculating section 34 calculates the pixel 
shift evaluation value and the reliability data of the image 
data, for every operation area of the image data, based on 
the first image data and the second image data (image data 
before and after the pixel shift) stored in the memory group 

20 32 (step S6) , and output the pixel shift evaluation value 
and the reliability data to the system controller 30. The 
system controller 30 judges whether the pixel shift 
photography has been performed normally, based on the pixel 
shift evaluation value and the reliability data of the image 

25 data calculated by the pixel shift evaluation value and 



reliability calculating section 34 (stepST). Thereafter, 
the system controller 30 displays the judgment result of 
the pixel shift photography on the display screen in the 
camera display section 7 (step S8). 

Fig. 3 and Fig. 4 show display examples of the camera 
display section 7. When judging that the pixel shift 
photography has been performed normally, the system 
controller 30 displays, as shown in Fig. 3, for example, 
"Pixel shift photographing has been performed normally", 
on the display screen in the camera display section 7. On 
the other hand, judging that the pixel shift photography 
has not been performed normally, the system controller 30 
displays, as shown in Fig. 4, for example, "Something is 
wrong in the pixel shift photography", on the display screen 
in the camera display section 7. 

The method of calculating the pixel shift evaluation 
value by the pixel shift evaluation value and reliability 
calculating section 34 in the above step S5 will now be 
described in detail. As the pixel shift evaluation value, 
(1) the coincidence degree of the image and (2) the amount 
of shift of the image can be used. A user can set which 
of (1) the coincidence degree of the image or (2) the amount 
of shift of the image is used as the pixel shift evaluation 
value, by a button in the camera operation section 8. 
(1) Method of calculation of the coincidence degree of 



the image : 

The method of calculating the coincidence degree of 
the image by the pixel shift evaluation value and reliability 
calculating section 34 will be described in detail with 
5 reference to Fig. 5A and Fig. 5B. 

Fig. 5A shows a photographed image 1 photographed 
before the shift of the image sensing device. Fig. 5B shows 
a photographed image 2 photographed after the shift of the 
image sensing device (after the image sensing device has 

10 been shifted by one pixel in the longitudinal direction) , 
and Fig. 5C shows a target image in the case where an ideal 
pixel shift has been performed . This target image indicates 
an image ideally shifted by one pixel in the longitudinal 
direction from the photographed image 1 . The pixel shift 

15 evaluation value and reliability calculating section 34 
produces the target image ideally shifted by one pixel in 
the longitudinal direction from the photographed image 1. 

Then, the pixel shift evaluation value and reliability 
calculating section 34 calculates the coincidence degree 

20 between the photographed image 2 and the target image . When 
the coincidence degree is "1", it can be judged that the 
ideal pixel shift has been performed. When the coincidence 
degree is abnormal, the coincidence degree decreases 0.99, 
0.98, ... depending on the abnormal degree. 

25 If this target image and the photographed image 2 



coincide with each other completely, that is, when the 
coincidence degree is "1", it can be said that ideal pixel 
shift photography has been performed. The correlation 
coefficient between these images is calculated as the 
5 coincidence degree of these two images. The correlation 
coefficient r can be calculated by using a formula by, for 
example, a pattern matching method as described below. 

nJ] IM - I)^ M 

10 

In the equation (1), M denotes a reference image and 
I denotes a comparison image. The correlation coefficient 
herein is calculated using the pattern matching method, but 
other methods may be used. 

15 In the pattern matching method, the ideal state is 

the coincidence degree = 1, and as the coincidence degree 
decreases, the state becomes far from the ideal state . With 
the pattern matching method, in the case where the 
coincidence degree is not "1", it cannot be judged how much 

20 degree is abnormal in which direction of the longitudinal 
direction or the lateral direction, but there is a merit 
in that the operation time is relatively short. 

(2) Method of calculation of the amount of shift of the 
image : 



The method of calculating the amount of shift of the 
image by the pixel shift evaluation value and reliability 
calculating section 34 will be described with reference to 
Fig. 6A to Fig. 11. The method of calculating the amount 
5 of shift can judge abnormality in the direction of the pixel 
shift and in the amount. 

Fig. 6A shows a photographed image 1 photographed 
before the shift of the image sensing device, and Fig. 6B 
shows a photographed image 2 photographed after the shift 

10 of the image sensing device. 

Of the photographed image 1 and the photographed image 
2 before and after the shift of the image sensing device, 
either one is fixed, and the other is shifted in the operation 
(not shifted physically, but imaginary shift is performed 

15 in the operation) . By calculating the mutual correlation 
coefficient, it is calculated how much and which direction 
of the shift makes the correlation coefficient largest. 

Fig. 7A to Fig. lOD show one example of the relation 
between the amount of shift in the X direction (lateral 

20 direction) and the Y direction ( longitudinal direction) and 
the correlation coefficient of the shifted image and the 
photographed image 1, when the photographed image 2 is 
shifted in the X direction (lateral direction) and the Y 
direction (longitudinal direction) . The value of the 

25 correlation coefficient in the figure and in the description 



below is not an actually calculated value, but a numerical 
value exaggerated for explanation. 

Fig. 7A shows a case where the photographed image 2 
is shifted by one pixel in the plus direction in the X direction 
5 (lateral direction) , and the correlation coefficient with 
respect to the photographed image 1 in this case becomes 
"0.45". Fig. 7B shows a case where the photographed image 
2 in Fig. 7A is shifted by one pixel in the minus direction 
in the Y direction (longitudinal direction) , and the 

10 correlation coefficient with respect to the photographed 
image 1 in this case becomes "0.7". Fig. 7C shows a case 
where the photographed image 2 in Fig. 7B is shifted by one 
pixel in the minus direction in the Y direction (longitudinal 
direction) , and the correlation coefficient with respect 

15 to the photographed image 1 in this case becomes "0.65". 
Fig. 7D shows a case where the photographed image 2 in Fig. 
7C is shifted by one pixel in the minus direction in the 
Y direction (longitudinal direction), and the correlation 
coefficient with respect to the photographed image 1 in this 

20 case becomes "0.4". 

Fig. 8A shows a case where the photographed image 2 
is shifted by 0 pixel in the plus direction in the X direction 
(lateral direction) , and the correlation coefficient with 
respect to the photographed image 1 in this case becomes 

25 "0.7". Fig. 8B shows a case where the photographed image 



2 in Fig. 8A is shifted by one pixel in the minus direction 
in the Y direction (longitudinal direction) , and the 
correlation coefficient with respect to the photographed 
image 1 in this case becomes "0.95". Fig. 8C shows a case 
5 where the photographed image 2 in Fig. 8B is shifted by one 
pixel in the minus direction in the Y direction (longitudinal 
direction) , and the correlation coefficient with respect 
to the photographed image 1 in this case becomes "0.9". Fig . 
8D shows a case where the photographed image 2 in Fig. 8C 

10 is shifted by one pixel in the minus direction in the Y 
direction (longitudinal direction) , and the correlation 
coefficient with respect to the photographed image 1 in this 
case becomes "0.65". 

Fig. 9A shows a case where the photographed image 2 

15 is shifted by one pixel in the minus direction in the X 
direction (lateral direction) , and the correlation 
coefficient with respect to the photographed image 1 in this 
case becomes "0.6". Fig. 9B shows a case where the 
photographed image 2 in Fig. 9A is shifted by one pixel in 

20 the minus direction in the Y direction (longitudinal 
direction) , and the correlation coefficient with respect 
to the photographed image 1 in this case becomes "0.85". 
Fig. 9C shows a case where the photographed image 2 in Fig. 
9B is shifted by one pixel in the minus direction in the 

25 Y direction (longitudinal direction) , and the correlation 



coefficient with respect to the photographed image 1 in this 
case becomes "0.8". Fig. 9D shows a case where the 
photographed image 2 in Fig. 9C is shifted by one pixel in 
the minus direction in the Y direction (longitudinal 
5 direction) , and the correlation coefficient with respect 
to the photographed image 1 in this case becomes "0.55". 

Fig. lOA shows a case where the photographed image 
2 is shifted by two pixels in the minus direction in the 
X direction (lateral direction) , and the correlation 

10 coefficient with respect to the photographed image 1 in this 
case becomes "0.3". Fig. lOB shows a case where the 
photographed image 2 in Fig. lOA is shifted by one pixel 
in the minus direction in the Y direction (longitudinal 
direction) , and the correlation coefficient with respect 

15 to the photographed image 1 in this case becomes "0 . 6" . Fig. 
IOC shows a case where the photographed image 2 in Fig. lOB 
is shifted by one pixel in the minus direction in the Y 
direction (longitudinal direction) , and the correlation 
coefficient with respect to the photographed image 1 in this 

20 case becomes "0.55". Fig. lOD shows a case where the 
photographed image 2 in Fig. IOC is shifted by one pixel 
in the minus direction in the Y direction (longitudinal 
direction) , and the correlation coefficient with respect 
to the photographed image 1 in this case becomes "0.25". 

25 Fig. 11 shows a graph in which the correlation 



coefficient with respect to the amount of shift in the X 
direction (lateral direction) and the Y direction 
(longitudinal direction) is plotted. The pixel shift 
evaluation value and reliability calculating section 34 
calculates the correlation coefficient with the 
interpolation operation up to a unit of 0.01 pixel, to thereby 
calculate the amount of shift in the lateral direction and 
in the longitudinal direction, when the correlation 
coefficient becomes largest. 

In the above example, the description has been made 
for a case where the photographed image 1 is fixed, and the 
photographed image 2 is shifted, but the photographed image 
2 is fixed and the photographed image 1 may be shifted. In 
this case, only the symbol of the calculated amount of shift 
is changed. 

With the above method of calculating the amount of 
shift, two-dimensional interpolation operation is 
performed, taking the longitudinal direction and the lateral 
direction into consideration, by shifting the image in the 
two directions, that is, the longitudinal direction and the 
lateral direction. Therefore, the operation time becomes 
long, but the direction and amount of the shift can be 
calculated, which is convenient for various judgment. 

In order to reduce the operation time, there can be 
considered such a simple method that the lateral direction 



(the direction originally not shifted) is ignored, and only 
the longitudinal direction is calculated . However, in this 
case, it is assumed that the amount of shift in the lateral 
direction is always within a certain range, and movement 
5 of hands in the lateral direction cannot be detected. 

Fig. 12 shows one example of the operation area in 
the image frame. In the example shown in this figure, the 
center of the screen and the peripheral areas (central area, 
left upper area, upper area, right upper area, left area, 

10 right area, left lower area, lower area and right lower area) 
are set as the operation area. The pixel shift evaluation 
value and reliability calculating section 34 calculates the 
amount of shift and the coincidence degree separately for 
each operation area. The pixel shift evaluation value and 

15 reliability calculating section 34 also calculates the 
reliability data indicating the reliability of the amount 
of shift and the coincidence degree separately for each 
operation area. 

The photographed image before and after the shift of 

20 the image sensing device and the amount of shift in each 
operation area in the photographed image will be described. 
Fig. 13A to Fig. 16B are diagrams for explaining the relation 
between the photographed image before and after the shift 
of the image sensing device and the amount of shift in each 

25 operation area. 



The judgment standard for judging normal or abnormal 
pixel shift photography in the system controller 30 will 
be described. In the case of the high resolution picture 
mode, it is judged that the pixel shift photography has been 
performed normally in the case where the amount of shift 
is within 1 + 0.1 pixel in the longitudinal and 0 ± 0 . 1 pixel 
in the lateral direction (within a predetermined range) . 
On the other hand, in the case of the normal resolution picture 
mode, it is judged that the pixel shift photography has been 
performed normally in the case where the amount of shift 
is within 1 ± 0.3 pixel in the longitudinal and 0 ± 0.3 pixel 
in the lateral direction (within a predetermined range) . 
In this manner, by calculating the amount of shift for each 
operation area, much information can be obtained compared 
to the method of calculating one amount of shift and amount 
of shift from the whole screen, thereby enabling fine 
j udgment . 

Fig. 13A and Fig. 13B show a case where a normal pixel 
shift has been performed, and the amount of shift is within 
1 ± 0.1 pixel in the longitudinal and 0 ± 0.1 pixel in the 
lateral direction in each operation area. The right lower 
operation area has a low contrast of the subject, and hence 
the reliability as the data is low. Moreover, this area 
includes a large probability of an error due to the low 
reliability. Therefore, it is not adequate as the data used 

39 



for the judgment of normal or abnormal pixel shift 
photography. In order to exclude such data, by taking into 
consideration not only the amount of shift but also the 
reliability data in each operation area, it is judged whether 
5 the pixel shift photography has been performed normally. 
Specifically, at the time of judging the pixel shift 
photography, the system controller 30 does not use the amount 
of shift in the operation area having low reliability data, 
or reduces weighting of the amount of shift in this operation 

10 area (in the case where weighting operation is performed) . 

Fig. 14A, Fig. 14B, Fig. 15A, and Fig. 15B show a case 
of movement of hands or an abnormal situation of the pixel 
shift unit 20 . If a phenomenon such as too much or too little 
longitudinal shift, or a lateral shift or a rotational shift 

15 occurs, as shown in Fig. 14A, Fig. 14B, Fig. 15A, and Fig. 
15B, the data related to the amount of shift in the operation 
area becomes such that a predetermined relation is maintained, 
and a part of or the whole of the amount of shift becomes 
without the predetermined range. 

20 The system controller 30 analyzes the data relating 

to the amount of shift data in each operation area in detail, 
to thereby judge if it is too much or too little longitudinal 
shift, a lateral shift, a rotational shift or a composite 
shift of these shifts . Here, it is difficult to judge whether 

25 the cause of this abnormal shift is due to movement of hands. 



or a failure of the pixel shift unit 20. 

Presumptively, if it is obvious, from the property 
of the pixel shift unit 20, that "even if the pixel shift 
unit 20 has a failure, a lateral shift or a rotational shift 
5 does not occur", it may be possible to judge that it is due 
to "a failure or movement of hands", only in the case of 
too much or too little longitudinal shift, and that it is 
due to movement of hands in the case of the lateral shift 
and the rotational shift . However, as described below, with 

10 regard to the "failure judgment", "when similar abnormal 
situation occurs substantially continuously", it is judged 
as a failure. 

Fig. 16A and Fig. 1 6B show a case where even if there 
is no movement of hands and the pixel shift unit 20 is normal, 

15 pixel shift photography becomes abnormal due to movement 
of the subject, and in particular, a case where a part of 
the data relating to the amount of shift in the operation 
area does not maintain a predetermined relation. In the 
example shown in this figure, the amount of shift in the 

20 central area and the right area are obviously abnormal 
compared to the amount of shift in other seven operation 
areas (left upper, upper, right upper, left, left lower, 
lower and right lower areas), and hence it can be judged 
that the subjects in these portions (in the central area 

25 and the right area) have moved. 



That is to say, the system controller 30 judges that 
the pixel shift photography has been performed normally, 
when the whole image is shifted normally (when the amount 
of shift is within the allowable range) , based on the amount 
5 of shift in each operation area in the image frame. 

Moreover, when the amount of shift in the operation 
area is not normal partly or wholly (when the amount of shift 
is without the allowable range) , and when the whole image 
is shifted, keeping a certain relation, the system controller 
10 30 judges that the amount of shift is abnormal due to movement 
of hands or a problem of the pixel shift unit 20. 

When the amount of shift in the operation area is not 
normal partly or wholly (when the amount of shift is without 
the allowable range) , and when there is no certain relation 
15 mutually (when only a part is abnormally shifted) , the system 
controller 30 judges that the pixel shift photography is 
abnormal due to movement of the subject. 

The system controller 30 may display the cause of the 
abnormal situation of the pixel shift photography (movement 
20 of hands, problem of the pixel shift unit 20, movement of 
the subject) , together with the display that the pixel shift 
photography 7 is abnormal, as described above. 

Description has been given for the amount of shift 
here, but it is the same for the case of the amount of shift. 
25 When it is necessary to reduce the operation time, only a 



part of the image (normally, the central portion) may be 
picked up for operation. In this case, the operation time 
can be made shorter, as the image area to be operated becomes 
smaller, but the accuracy decreases. Hence, a certain 
5 extent is required. 

The method for judging the reliability evaluation of 
the image obtained by the pixel shift photography by the 
pixel shift evaluation value and reliability calculating 
section 34 will now be described in detail. The pixel shift 

10 evaluation value and reliability calculating section 34 
calculates the reliability data for evaluating the 
reliability of the calculated pixel shift evaluation value 
(the amount of shift and the coincidence degree) . 

Specifically, the pixel shift evaluation value and 

15 reliability calculating section 34 detects the contrast of 
the image in each operation area in the image frame to 
calculate the reliability data corresponding to the contrast 
For example, the reliability data is an evaluation value 
indicating that the reliability becomes higher with an 

20 increase of the contrast, as five-stage evaluation values. 
That is to say, since a subject having a low contrast (= 
a monotonous wall , etc.) has little information amount, the 
calculation error of the amount of shift and the coincidence 
degree increases. Therefore, a portion having a contrast 

25 less than a certain degree is not used for the operation. 



or weighting therefor is made low. Here, the reliability 
data is calculated based on the contrast of the image, but 
the reliability data may be calculated based on minuteness 
of the image (frequency) , brightness of the image, 
repeatability of the image and the like, other than the 
contrast . 

As described above, according to the first embodiment, 
in the pixel shift photography, the pixel shift evaluation 
value and reliability calculating section 34 calculates a 
pixel shift evaluation value (the amount of shift or the 
coincidence degree) based on the image data photographed 
before and after the shift of the image sensing device, and 
the system controller 30 judges the result of pixel shift 
photography based on the pixel shift evaluation value. 
Hence, a failure in the pixel shift photography caused by 
movement of hands, movement of the subject, or a problem 
of the pixel shift mechanism can be informed to the 
photographer, thereby the photographer can obtain a desired 
pixel shifted image by re-photographing or the like. 

Also, according to the first embodiment, the pixel 
shift evaluation value and reliability calculating section 
34 calculates the amount of shift between the photographed 
images before and afterthe shift of the image sensing device, 
and the system controller 30 judges whether the pixel shift 
photography has been performed normally, based on the amount 



of shift between the photographed images before and after 
the shift of the image sensing device. As a result, it 
becomes possible to j udge whether the pixel shift photography 
is successful or not. 
5 Moreover, according to the first embodiment , the pixel 

shift evaluation value and reliability calculating section 
34 calculates a pixel shift evaluation value (the amount 
of shift or the coincidence degree) for each operation area 
in the image frame. Hence, it can be detected whether the 
10 pixel shift evaluation value (the amount of shift or the 
coincidence degree) is adeguate or not in a plurality of 
points of the image, and as a result, more accurate judgment 
becomes possible with regard to the shift degree of the whole 
image . 

15 More specifically, when the whole image is normally 

shifted, it is judged that the pixel shift photography has 
been performed normally, and when a part or the whole of 
the image is not normal, and the whole image is shifted, 
keeping a certain relation, it is judged to be abnormal due 

20 to a problem of the pixel shift unit 20. When a part of 
or the whole of the image is not normal, and there is no 
certain relation mutually (when only a part of the image 
is abnormally shifted) , it is judged that the pixel shift 
photography is abnormal due to movement of the subject. As 

25 a result, it becomes possible to judge the reason of 



abnormality in the pixel shift photography. 

According to the first embodiment, the pixel shift 
evaluation value and reliability calculating section 34 
calculates the reliability data for evaluating the 
5 reliability of the calculated pixel shift evaluation value 
(the amount of shift and the coincidence degree) , and the 
system controller 30 calculates the reliability data based 
on the amount of shift in each operation area, taking the 
reliability data into consideration, at the time of judgment 

10 of the pixel shift photography. Hence, at the time of 
judgment of the pixel shift photography, the pixel shift 
evaluation value in the operation area having a large 
influence of noise is not used, or weighting therefor can 
be made small. As a result, erroneous judgment of the pixel 

15 shift photography can be prevented. 

According to the first embodiment, since the 
reliability data is also calculated based on the contrast 
information of the subject (image), the pixel shift 
evaluation value in the operation area having a low contrast, 

20 and hence having a large probability of an error can be taken 
into account, thereby enabling more accurate judgment of 
the pixel shift. 

Furthermore, according to the first embodiment, since 
the judgment result of the pixel shift photography is 

25 displayed in the camera display section 7, the photographer 



is informed of abnormal pixel shift photography . As a result, 
the photographer can take suitable measures (measures for 
preventing movement of hands or restricting movement of the 
subject, making a service call, etc.) . 

A digital camera according to the second embodiment 
will now be described with reference to Fig. 17 and Fig. 
18. In the digital camera in the second embodiment, it is 
judged whether the pixel shift unit 20 has a failure or not 
based on the data of the pixel shift photography performed 
before (pixel shift photography relational data) , and an 
adjustment value for driving the pixel shift unit 20 is 
automatically changed. The function for automatically 
changing the adjustment value for driving the pixel shift 
unit 20 is referred to herein as a "self calibration 
function" . 

The digital camera according to the second embodiment 
has a construction that is similar to that of the digital 
camera in the first embodiment (see Fig. 1) , and hence only 
the different point between these will be described here. 

In addition to the photographed image data, the 
adjustment value for driving the pixel shift unit 20 and 
the pixel shift photography relational data (pixel shift 
evaluation value (the amount of shift or the coincidence 
degree) and the reliability data) are stored in the memory 
group 32. The system controller 30 stores, in the memory 



group 32, the pixel shift photography relational data (the 
pixel shift evaluation value and the reliability data) 
calculated by the pixel shift evaluation value and 
reliability calculating section 34, every time the pixel 
5 shift photography is performed. 

The system controller 30 also judges whether the pixel 
shift unit 20 has a failure or not, based on the pixel shift 
photography relational data (the pixel shift evaluation 
value, the reliability data, etc.) obtained by performing 

10 the pixel shift photography for several times, which is 
stored in the memory group 32. Specifically, the system 
controller 30 judges that the pixel shift unit 20 has a 
"failure", in the case where "abnormality having a similar 
tendency occurs substantially continuously, for a 

15 predetermined number of times" , in the pixel shift evaluation 
value. For example, assuming a case where abnormality 
having another tendency happens to occur due to movement 
of hands, the system controller 30 judges as a failure of 
the pixel shift unit 20, when "the same problem occurred 

20 6 times or more in the last ten cases". In the case where 
a failure in the pixel shift unit 20 is judged by using the 
coincidence degree, it is difficult to judge whether it is 
"the same tendency" or not. Hence, the accuracy in the 
failure judgment may decrease, however, the operation time 

25 can be made short. Therefore, the method is selected 



depending on which is given priority, time or accuracy. 

Moreover, when an instruction to start the pixel shift 
photography is input, the system controller 30 executes the 
self calibration mode, in which the adjustment value for 
driving the pixel shift unit 20 is automatically changed. 

In the camera operation section 8, the photographing 
frequency (predetermined number) in the pixel shift 
photography relational data (the pixel shift evaluation 
value, the reliability data, etc.) stored in the memory group 
32 can be set. For example, when "5" is set as the 
photographing frequency, the pixel shift photography 
relational data calculated at the time of the pixel shift 
photography of the last five times is stored in the memory 
group 32. The data before that is deleted. The camera 
operation section 8 can change the adjustment value stored 
in the memory group 32 to a default value. 

The pixel shift photography operation of the digital 
camera according to the second embodiment will now be 
described with reference to the flowchart in Fig. 17. Fig. 
17 is a flowchart for explaining the operation of pixel shift 
photography. When an instruction to start the pixel shift 
photography is input from the camera operation section 8 
(step Sll), the system controller 30 judges whether there 
is a failure in the pixel shift unit 20 or not (step S12), 
based on the pixel shift photography relational data (the 



pixel shift evaluation value and the reliability data) of 
the last several times, which is stored in the memory group 
32. When there is a failure in the pixel shift unit 20 (step 
S13), control proceeds to step S23, to thereby display in 
the camera display section 7 that there is a failure in the 
pixel shift unit 20. 

Fig. 18 shows a display example of the camera display 
section 7 . When the system controller 30 judges that there 
is a failure in the pixel shift unit, as shown in Fig. 18, 
displays "The pixel shift unit has a failure" on the display 
screen in the camera display section 7 . 

On the other hand, in step S13, when there is no failure 
in the pixel shift unit 20, control proceeds to step S14, 
and the system controller 30 judges that it is not a failure 
of the pixel shift unit 20, but if it is necessary to change 
the adjustment value for driving the pixel shift unit 20 
or not (step S14), based on the pixel shift photography 
relational data (the pixel shift evaluation value and the 
reliability data) of the last several times stored in the 
memory group 32 . This is because a case is taken into account, 
where it cannot be said as a failure of the pixel shift unit 
20, but the pixel shift is not performed normally. 

As a result of this judgment, when it is judged that 
a change of the adjustment value is not necessary, control 
proceeds to step S16, and when it is judged that a change 



of the adjustment value is necessary, control proceeds to 
step S15. 

In step S15, the system controller 30 calculates a 
new adjustment value based on the pixel shift evaluation 
value of the last several times, which is stored in the memory 
group 32, and changes the adjustment value stored in the 
memory group 32 to the calculated new adjustment value, and 
control proceeds to step S16. 

In step S16, the first photographing is performed, 
and the image data is stored in the memory group 32. The 
system controller 3 0 then outputs the adjustment value stored 
in the memory group 32 to the pixel shift drive circuit 6 
to drive the pixel shift unit 20 via the pixel shift drive 
circuit 6, to thereby shift the image sensing device by one 
pixel in the longitudinal direction (pixel shift) (stepS17) . 
In a state with the image sensing device shifted by one pixel 
in the longitudinal direction, the second photographing is 
performed, and the image data is stored in the memory group 
32 (step S18). The image processing section 33 combines 
the first image data and the second image data stored in 
the memory group 32 , to thereby generate the image data having 
a high resolution (step S19) . After completion of the pixel 
shift photography, the pixel shift evaluation value and 
reliability calculating section 34 calculates the pixel 
shift evaluation value and the reliability data of the image 



data, for each operation area in the image frame (see Fig. 
12) , based on the first image data and the second image data 
(image data before and after the pixel shift) stored in the 
memory group 32 (step S20), and outputs these data to the 
5 system controller 30. The system controller 30 judges 
whether the pixel shift photography has been normally 
performed or not based on the pixel shift evaluation value 
and the reliability data of the image data calculated by 
the pixel shift evaluation value and reliability calculating 

10 section pixel shift evaluation value and reliability 
calculating section 34 (step S21) . Thereafter, the system 
controller 30 displays the judgment result of the pixel shift 
photography on the display screen in the camera display 
section 7 (step S22) . The display example of the judgment 

15 result is similar to the figure. 

The self calibration function of the above adjustment 
value (the above step Sl5) will now be described. As the 
method of calculating the adjustment value, there are two 
methods, (1) a method of calculating the adjustment value 

20 based on the data relating to the amount of shift, and (2) 
a method of calculating the adjustment value based on the 
coincidence degree. Which of (1) the amount of shift of 
the image and (2) the coincidence degree of the image is 
to be usedmay be fixed by the concept of the camera, depending 

25 on which is given priority, the processing speed or the 



accuracy, or may be chosen by a user using a button in the 
camera operation section 8 . In either case , it is necessary 
to store the adjustment value used at the time of performing 
the pixel shift photography for each time in the camera. 

(1) Method of calculating the adjustment value based 
on the data relating to the amount of shift 

In the ROM of the system controller 30, there is stored 
an adjustment value - amount of shift table in which 
relational data indicating the relation (inclination) 
between the adjustment value and the amount of shift is stored, 
When judging that the amount of shift of the pixel shift 
photography of the last time is not within the allowable 
range, the system controller 30 calculates a new adjustment 
value based on the pixel shift photography relational data 
(the amount of shift, the reliability data, etc.) of the 
last time stored in the memory group 32, referring to the 
adjustment value - amount of shift table in the ROM. 

Since there can be considered that the relation between 
the adjustment value and the amount of shift may include 
a mechanical difference, the accuracy of the calculated new 
adjustment value may be slightly rough. However, by 
repeating the same thing, the adjustment value will be 
settled to an optimum adjustment value. Repeating is 
important. Therefore, if it is made easy to repeat self 
calibration, such that after self calibration is once 



performed by problem j udgment , problem j udgment i s performed 
under lighter conditions than before, then the adjustment 
value will be settled earlier. 

(2) Method of calculating the adjustment value based 
5 on the coincidence degree 

In the ROM of the system controller 30, there is stored 
an adjustment value - coincidence degree table (see Fig. 
25) in which relational data indicating the relation 
(inclination) between the adjustment value and the 

10 coincidence degree is stored. When judging that the 
coincidence degree of the pixel shift photography of the 
last time is not within the allowable range, the system 
controller 30 calculates a new adjustment value based on 
the pixel shift photography relational data (the coincidence 

15 degree, the reliability data, etc. ) of the last time stored 
in the memory group 32, referring to the adjustment value 
- coincidence degree table in the ROM. 

Whether the adjustment value is to be increased or 
decreased cannot be judged from these information. 

20 Therefore, there can be considered a method in which it is 
set to be increased (or decreased) always, overlooking a 
slight sacrifice, and when it is judged that "it should be 
decreased (increased)", by referring to the next 
photographing result, the setting is readjusted. 

25 In view of the nature of the mechanism in the pixel 



shift unit 20, if there is a tendency that an abnormality 
is likely to occur (for example, there is a general tendency 
such that the pixel amount of shift gradually increases with 
the lapse of time) , it is desired that increase or decrease 
5 is matched to the tendency. This method has a disadvantage 
in that there is a possibility that the image quality may 
further decrease temporarily than before, but also has an 
advantage in that the operation can be performed faster than 
the amount of shift calculation method. In this case, it 

10 is considered that the accuracy of the new adjustment value 
may be considerably rough than in the amount of shift 
calculation method, but by repeating the similar thing, it 
is possible that the adjustment value is settled to the 
optimum adjustment value. Repeating is important. 

15 Therefore, if it is made easy to repeat self calibration, 
such that after self calibration is once performed by problem 
judgment, problem judgment is performed under lighter 
conditions than before, then the adjustment value will be 
settled earlier. 

20 The self calibration function is for automatically 

correcting the adjustment value, and hence can create the 
best pixel shifted image all the time, but there is a problem 
in that when the use environment is temporarily changed, 
how it is to be dealt with. For example, there can be 

25 mentioned a case of "temperature characteristic" . In the 



case where a camera normally used in a normal temperature 
is suddenly to be used in a cold place, due to the temperature 
characteristic of the pixel shift mechanism, there is a 
possibility that the amount of shift of the pixel may not 
5 be optimum. If the above-described self calibration is 
performed at this time, on the contrary, adequate amount 
of shift of the pixel cannot be obtained when the use 
environment returns to the normal temperature, and normal 
pixel shift photography may not be performed. Therefore, 

10 in the case where an abrupt temperature change may exist, 
the camera operation section 8 is operated to return the 
adjustment value to the default value or the value before 
the adjustment, thereby such a problem can be solved. 

Moreover, in the case of such an environment change, 

15 by reducing the frequency of pixel shift photography used 
for the judgment for self calibration (for example, up to 
two to three times), self calibration starts earlier, and 
as a result, photographing error can be reduced. 

On the other hand, in the case where the camera is 

20 used continuously in a stable environment, if calibration 
is performed too frequently, disturbance by the movement 
of hands may be picked up . Therefore, it is considered that 
the necessary number of times is about five. Then, it is 
desirable that the photographer can properly change the 

25 number of data used for the pixel shift problem judgment. 



In this case, a method of inputting the frequency itself 
may be considered. However, selection from about three 
stages will be sufficient. 

According to the digital camera in this embodiment, 
5 since it is considered that proper calibration is possible 
after shipment, the adjustment accuracy in the factory can 
be made rough, thereby enabling simplification of the 
adjustment step. Because, in the case of a conventional 
camera of a type that cannot be calibrated after shipment, 

10 adjustment accuracy close to zero is required, taking into 
account a change with the lapse of time and a fluctuation 
portion due to the temperature characteristic. 

As described above, according to the second embodiment, 
the system controller 30 judges whether the pixel shift unit 

15 20 has a failure or not, based on the pixel shift photography 
relational data obtained by performing the pixel shift 
photography for several times . Therefore, when an abnormal 
situation occurs in the pixel shift photography, it is 
possible to judge whether it is temporary or permanent 

20 ( = failure) . As a result, it becomes possible to judge a 
failure of the pixel shift unit 20. 

According to the second aspect, the system controller 
30 judges whether the pixel shift unit 20 has a failure or 
not, based on the amount of shift obtained by performing 

25 the pixel shift photography for several times, thereby 



enabling judgment of a failure of the pixel shift unit 20 
accurately . 

According to the second aspect, the pixel shift 
evaluation value and reliability calculating section 34 
5 calculates the pixel shift evaluation value (the amount of 
shift or the coincidence degree) for each operation area 
of the image frame. As a result, it can be detected whether 
the pixel shift evaluation value (the amount of shift or 
the coincidence degree) is adequate or not in a plurality 

10 of points in the image, thereby enabling accurate judgment 
with regard to the shift degree of the whole image. 

According to the second aspect, the pixel shift 
evaluation value and reliability calculating section 34 also 
calculates the reliability data for evaluating the 

15 reliability of the calculated pixel shift evaluation value 
(the amount of shift and the coincidence degree) , and the 
system controller 30 takes the reliability data into account 
at the time of judgment of the pixel shift photography to 
thereby calculate the reliability data based on the amount 

20 of shift in each operation area. As a result, at the time 
of judgment of a failure of the pixel shift unit 20, noise 
can be made minimum, thereby enabling judgment of a failure 
of the pixel shift unit 20 accurately. 

According to the second embodiment, this reliability 

25 data is calculated based on the contrast information of a 



subject (an image) , hence the pixel shift evaluation value 
in the operation area having a low contrast and hence having 
a large probability of an error can be taken into account. 
As a result, a failure of the pixel shift unit 20 can be 
5 judged highly accurately. 

According to the second embodiment, when judging that 
the pixel shift unit 20 has a failure, the system controller 
30 displays that the pixel shift unit 20 has a failure in 
the camera display section 7. As a result, the photographer 

10 can take suitable measures (suspension of use of the pixel 
shift mode, making a service call, etc.) . 

According to the second embodiment, when there is a 
slight failure in the pixel shift mechanism, the adjustment 
value for the pixel shift drive can be automatically changed 

15 (=self calibration) , hence it becomes possible to return 
to the normal condition. As a result, a slight failure can 
be restored without making a service call, thereby enabling 
a reduction of unsuccessful pixel shift photography as low 
as possible. 

20 According to the second embodiment, the camera 

operation section 8 is operated to enable resetting of the 
adjustment value to the default value, hence adequate pixel 
shift can be performed even at the time of a temporary use 
(for example, use at a low temperature temporarily), and 

25 in the normal condition, the adjustment value is returned 



to the original adjustment value to perforin adequate pixel 
shift photography. 

According to the second embodiment, a user can operate 
the camera operation section 8 to set the photographing 
frequency in the pixel shift photography relational data 
of the past pixel shift photography to be stored in the memory 
group 32 (photographing frequency for self calibration) . 
Hence, when the photographer decides to use the camera in 
a special environment temporarily, the photographing 
frequency for calibration can be set to a smaller value, 
to perform self calibration within a short period of time 
intentionally. Specifically, if the photographer sets the 
photographing frequency for self calibration 
(photographing frequency in the pixel shift photography 
relational data to be stored in the memory group 32) to 1 
to 2 times, when the photographing environment changes from 
the previous condition (for example, in the case where the 
photographer goes out to the ski field from the room, or 
where photographing is started from a moving train) , the 
self calibration mode works immediately, and adequate pixel 
shift photography becomes possible from the second to the 
third shots. That is to say, when the photographing 
environment changes , if the photographing frequency for self 
calibration is set to a large value, the pixel shift 
evaluation value at the time of performing the pixel shift 



photography in the previous environment affects largely at 
the time of calculating the adjustment value. Therefore, 
in order to calculate the adjustment value corresponding 
to the new environment, it is necessary to perform pixel 
5 shift photography many times. Hence, when the environment 
changes, it is desired to set the photographing frequency 
for self calibration to a small value. 

In the self calibration mode, a shift in the direction 
to be calibrated (in the longitudinal direction) is naturally 
10 necessary, but a shift in the lateral direction is hardly 
useful for calibration. Therefore, the pixel shift 
evaluation value and reliability calculating section 34 may 
ignore the lateral direction (the direction which is not 
moved naturally) , and calculate only the longitudinal 
15 direction, in order to reduce the calculation time. 

In order to reduce the data storage amount of the memory 
group 32, the pixel shift evaluation value may be corrected 
based on the reliability data, and the corrected pixel shift 
evaluation value may be stored in the memory group 32. 
20 A digital camera according to the third embodiment 

will now be described, with reference to Fig. 19 to Fig. 
25. The digital camera in the third embodiment comprises 
a self calibration mode for adjusting the adjustment value 
for driving the pixel shift unit 20, and the self calibration 
25 mode is executed in response to the instruction of a 



photographer , 

The digital camera in the third embodiment has a 
construction that is similar to that of the digital camera 
in the first embodiment (see Fig. 1) , and hence only the 
5 different point between these will be described here. 

The camera operation section 8 comprises a self 
calibration mode selection button for selecting the self 
calibration mode. When this self calibration mode 
selection button is pushed, the system controller 3 0 executes 

10 the self calibration mode. As the key operation for 
selecting the self calibration mode, any form may be used. 
For example, the self calibration mode may be selected from 
the operation menu. Alternatively, since the self 
calibration mode is a mode having a low use frequency, the 

15 self calibration mode may be executed when a plurality of 
buttons is pushed at the same time. Further, the self 
calibration mode may be executed, when after the self 
calibration mode selection button has been pushed, another 
key (release button and etc.) is operated. 

20 When the self calibration mode selection button in 

the camera operation section 8 is pushed thereby to input 
an instruction to start the self calibration mode from the 
camera operation section 8, the system controller 30 executes 
the self calibration mode and re-calculates an adjustment 

25 value for driving the pixel shift unit 20 and sets the 



calculated adjustment value (see Fig. 19). In this manner, 
in the self calibration mode , similar processingis performed 
to that for the adjustment (setting) of the adjustment value 
of the pixel shift unit 20 performed in a manufacturing plant . 
5 However, differing from the adj ustment in the manufacturing 
plant, since the self calibration mode has to be processed 
in the camera, it is required that the calibration time is 
short and the operation is easy. On the other hand, it is 
desired that the adjustment accuracy is as high as possible . 

10 However, since the photographer can execute the self 
calibration mode frequently depending on the environment, 
a slight adjustment error is allowable. 

The operation of the pixel shift photography of the 
digital camera according to the third embodiment will now 

15 be described with reference to the flowchart in Fig. 19. 
Fig. 19 shows a flowchart for explaining the operation of 
the pixel shift photography. 

The system controller 30 shown in Fig. 19 judges if 
an instruction to start the self calibration mode is input 

20 from the camera operation section 8 or not (step S31), and 
if the instruction to start the self calibration mode has 
not been input, control proceeds to step S35. On the other 
hand, if the instruction to start the self calibration mode 
has been input, the system controller 30 executes the self 

25 calibration mode to calculate the adjustment value (step 



S32). The specific method of calculating the adjustment 
value will be described later. 

The system controller 30 then judges if the calculated 
adj ustment value is within a predetermined range ornot (step 
5 S33), and if the adj ustment value is within the predetermined 
range, control proceeds to step S35. On the other hand, 
if the adjustment value is without the predetermined range, 
it is judged whether it is due to a failure of the pixel 
shift unit 20 or not. As a result of this judgment, if it 

10 is not because of a failure of the pixel shift unit 20, control 
proceeds to step S35. On the other hand, if it is judged 
that the pixel shift unit 20 has a failure, control proceeds 
to step S43, and the system controller 30 displays that the 
pixelshiftunit20hasafailureinthe camera display section 

15 7 . 

In step S35, it is judged if there is an instruction 
to start the pixel shift photography from the camera 
operation section 8 or not. If an instruction to start the 
pixel shift photography is received, control returns to step 

20 S31. If an instruction to start the pixel shift photography 
is not received, control proceeds to step S36. 

In step S36, the first photographing is performed and 
the image data is stored in the memory group 32. The system 
controller 30 then outputs the adjustment value to the pixel 

25 shift drive circuit 6, to drive the pixel shift unit 20 via 



the pixel shift drive circuit 6, to thereby shift the image 
sensing device by one pixel in the longitudinal direction 
(pixel shift) (step S37) . In a state with the image sensing 
device shifted by one pixel in the longitudinal direction, 
5 the second photographing is performed, and the image data 
is stored in the memory group 32 (step S38) . The image 
processing section 33 combines the first image data and the 
second image data stored in the memory group 32 to generate 
the image data having a high resolution (step S39) . After 
10 completion of the pixel shift photography, the pixel shift 
evaluation value and reliability calculating section 34 
calculates the pixel shift evaluation value and the 
reliability data of the image data, for each operation area 
(see Fig, 12) of the image data, based on the first image 
15 data and the second image data (image data before and after 
the pixel shift) stored in the memory group 32 (step S40), 
and outputs the pixel shift evaluation value and the 
reliability data to the system controller 30. The system 
controller 30 judges whether the pixel shift photography 
20 has been performed normally, based on the pixel shift 
evaluation value and the reliability data of the image data 
calculated by the pixel shift evaluation value and 
reliability calculating section 34 (stepS41). Thereafter, 
the system controller 30 displays the judgment result of 
25 the pixel shift photography on the display screen in the 



camera display section 7 (step S42). 

The method of calculating the adjustment value in the 
self calibration mode in the above step S32 will now be 
described in detail with reference to Fig. 20A to Fig. 
25. As the method of calculating the adjustment value, there 
are two methods, (1) a method of calculating the adjustment 
value based on the amount of shift data, and (2) a method 
of calculating the adjustment value based on the coincidence 
degree. Which of (1) the amount of shift of the image and 
(2) the coincidence degree of the image is to be used as 
the pixel shift evaluation value may be fixed by the concept 
of the camera, depending on which is given priority, the 
processing speed or the accuracy, or may be chosen by a user 
using a button in the camera operation section 8. 

(1) Method of calculating the adjustment value based on 
the coincidence degree: 

The method of calculating the adjustment value based 
on the coincidence degree will be described with reference 
to Fig. 2 OA to Fig . 22 . At first, the pixel shift evaluation 
value and reliability calculating section 34 sets a target 
image. Specifically, the pixel shift evaluation value and 
reliability calculating section 34 calculates, asthetarget 
image, an image obtained by shifting a photographed image 
1 photographed without shifting the image sensing device, 
as shown in Fig. 20A, by one pixel in the longitudinal 

66 



direction on the operation, as shown in Fig. 20B. 

The system controller 30 then changes the adjustment 
value adequately, and shifts the image sensing device by 
the amount, respectively, corresponding to the plurality 
of adjustment values, to execute photographing at each 
position of the image sensing device. The pixel shift 
evaluation value and reliability calculating section 34 
calculates the coincidence degree between each photographed 
image and the target image. The coincidence degree is 
calculatedby calculating the correlation coefficient using 
the pattern matching method as described above. Fig. 21A 
to Fig. 21H are diagrams for explaining the coincidence 
degree between the photographed image and the target image, 
when the adjustment value is changed. These figures show 
the photographed images when the adjustment value is set 
to be 80, 90, 100, 110, 120, 130, 140 and 150, and an example 
of the coincidence degree between the above photographed 
images 2 to 9 and the target image . 

Lastly, the system controller 30 calculates the 
adjustment value having the largest coincidence degree by 
the interpolation operation. Fig. 22 is a graph showing 
the relationbetween the ad j ustment value and the coincidence 
degree with respect to the target image, when the coincidence 
degree with respect to the target image is calculated by 
changing the adjustment value (adjustment value = 80, 90, 



100, 110, 120, 130, 140 and 150) . In this figure, the 
coincidence degree has a peak between the adjustment value 
of 100 to 110, and the optimum adjustment value becomes 104 
by the interpolation operation . According to such a method, 
5 a certain photographing frequency is necessary, but the 
operation itself is relatively simple. 

(2) Method of calculation of the adjustment value based 
on the amount of shift data: 

The method of calculating the optimum adjustment value 

10 with the amount of shift calculation method will be described 
with reference to Fig. 23 to Fig. 25. Fig. 23 is a diagram 
for explaining the calibration method of the adj ustment value 
using the amount of shift calculation method. In the 
calibration method that is based on the amount of shift 

15 calculation method, the adjustment value having the amount 
of shift of "1" is counted back. 

Fig. 23 and Fig. 24 are diagrams for explaining a case 
where the optimum adjustment value is calculated from 
adjustment values at two points. Fig. 23 and Fig. 24 show 

20 the relation between the adjustment value and the amount 
of shift in the longitudinal direction (pixel). Fig. 23 
shows a case where the adjustment value is relatively good, 
and Fig. 24 show a case where the adjustment value deviates 
largely . 

25 In Fig. 23, the image sensing device is shifted with 



an adjustment value (100 and 130 in this figure) before and 
after the previous adjustment value (115 in the example shown 
in this figure) to photograph each image, the amount of shift 
is respectively calculated (in the example shown in this 
figure, the amount of shift is "0.97" when the adjustment 
value is "100", and the amount of shift is "1.25" when the 
adjustment value is "130") , and the optimum adj us tment value 
is calculated from these two points by a proportional 
calculation. Specifically, the amount of shift is 
calculated in the longitudinal direction, when the 
adjustment value is 100 and 130, and the adjustment value 
when the longitudinal amount of shift becomes "1" by the 
proportional calculation is calculated, to thereby obtain 
the optimum adjustment value. 

Fig. 25 is a diagram for explaining a case where the 
optimum adjustment value is calculated from an adjustment 
value at one point, and showing the relation between the 
adjustment value and the amount of shift in the longitudinal 
direction (pixel) . An inclination line showing the average 
relation of the adjustment value and the amount of shift 
is stored in the ROM in the system controller 30, from the 
amount of shift with the previous adjustment value, and the 
optimum adjustment value having the amount of shift in the 
longitudinal direction of "1" is calculated from this 
inclination line showing the average relation of the 



adjustment value and the amount of shift. 

The method of calculating the optimum adjustment value 
from an adjustment value at one point has a merit in that 
the photographing frequency can be decreased, though the 
5 accuracy decreases as far as the inclination line and the 
shift degree exist, compared to the case where the optimum 
adjustment value is calculated from adjustment values at 
two points. The accuracy of the optimum adjustment value 
may be improved by repeating the method of calculating the 

10 optimum adjustment value from an adjustment value at one 
point or the method of calculating the optimum adjustment 
value from adjustment values at two points . 

In order to perform accurate calibration of the 
adjustment value, a subject most suitable for calibration 

15 may be used. Here, the most suitable subject stands for 
a subject having a certain contrast and adequate brightness, 
which is not a continuous pattern, and which is in focus. 
The system controller 30 judges the contrast, continuous 
pattern, brightness and focus of the image photographed in 

20 pre-photographing in the self calibration mode. When it 
is judged that it is not a subject most suitable for 
calibration, a warning message such as "Please change the 
subject" or "It is out of focus" may be displayed in the 
camera display section 7. 

25 In order to improve the calibration accuracy, a 



calibration chart recommended by the manufacturer, which 
is printed beforehand in the operation manual or on an 
individual package, may be used. 

As described above , according to the third embodiment, 
5 a self calibration mode is provided for changing the 
adjustment value for driving the pixel shift unit 20, and 
when the self calibration mode selection button in the camera 
operation section 8 is pushed, the self calibration mode 
is executed. Hence, the photographer can perform 

10 calibration of the adj ustment value anytime at his discretion, 
thereby the best pixel shifted image can be obtained always . 

According to the third embodiment, in the self 
calibration mode , the optimum adjustment value is calculated 
based on the subject image obtained by performing pre-pixel 

15 shift photography, and hence accurate calibration of the 
adjustment value can be performed. 

According to the third embodiment, by the pre-pixel 
shift photography in the self calibration mode, the amount 
of shift is calculated and the optimum adjustment value is 

20 calculated based on the calculated amount of shift. As a 
result, the photographing frequency required for 
calibration can be reduced. Specifically, pixel shift 
photography is performed with two adjustment values before 
and after the current adjustment value to calculate each 

25 amount of shift, and the calculated adjustment value is 



calculated by a proportional calculation. Hence, a 
relatively accurate adjustment value can be calculated with 
a fewer photographing frequency. Moreover, since the pixel 
shift photography is performed with the current adjustment 
value, and a new adjustment value is determined by a 
theoretical calculation based on the capacity data, the 
adjustment value can be adjusted at high speed. 

According to the third embodiment, in the self 
calibration mode, a plurality of pixel shift photography 
is performed, while changing the adj us tment value , and based 
on each amo-unt of shift, an adjustment value having a peak 
amount of shift is calculated as the optimum adj us tment value . 
As a result, though the photographing frequency increases, 
the peak in the focusing degree can be actually searched, 
and hence an adjustment value having higher accuracy can 
be obtained. 

According to the third embodiment, since the pixel 
shift evaluation value and reliability calculating section 
34 calculates the pixel shift evaluation value (the amount 
of shift or the coincidence degree) for each operation area 
in the image frame, it can be detected whether the pixel 
shift evaluation value (the amount of shift or the 
coincidence degree) in a plurality of points in the image 
is adequate or not. As a result, a precise adjustment value 
can be obtained with regard to the shift degree (coincidence 



degree) of the whole image. 

According to the third embodiment, since the 
reliability data is calculated based on the contrast 
information of the subject (image), the pixel shift 
evaluation value in the operation area having a low contrast, 
and hence having a large probability of an error can be taken 
into account, thereby enabling more accurate judgment of 
a failure of the pixel shift unit 20. 

According to the third embodiment, the system 
controller 30 judges the contrast, continuous pattern, 
brightness and focus of the image photographed in 
pre-photographing in the self calibration mode. When it 
is judged that it is not a subject most suitable for 
calibration, a message prompting a change of the subject 
is displayed in the camera display section 7. Hence, it 
can be informed to the photographer that this subject cannot 
be used for calibration, and an inadequate calibration can 
be prevented from being performed, thereby enabling 
prevention of unsuccessful pixel shift photography. 

According to the third embodiment, when the optimum 
adjustment value calculated in the self calibration mode 
is without an allowable range, it is judged to be a failure 
of the pixel shift unit 20 (a failure that cannot be restored 
by readjustment), hence pixel shift photography can be 
prevented from being performed in an adequate condition. 



As a result, the photographer can take suitable measures 
(suspension of use of the pixel shift mode, making a service 
call, etc . ) . 

A digital camera according to the fourth embodiment 
will nowbe described with reference to Fig. 26. The digital 
camera in the fourth embodiment has a normal pixel shift 
photography mode for performing normal pixel shift 
photography, and a continuous pixel shift photography mode 
in which when pixel shift photography is performed, pre -pixel 
shift photography is first performed, to calculate an 
adjustment value for driving the pixel shift unit 20, and 
pixel shift photography is performed based on the calculated 
adjustment value. 

The digital camera according to the fourth embodiment 
has a construction that is similar to that of the digital 
camera in the first embodiment (see Fig. 1) , and hence only 
the different point between these will be described here. 
The camera operation section 8 selects a normal pixel shift 
photography mode or a continuous pixel shift mode by a mode 
selection button thereby to output an instruction to start 
each mode to the system controller 30 . The camera operation 
section 8 also can set the upper limit of the frequency (a 
predetermined number of times A) for executing the pre-pixel 
shift photography in the continuous pixel shift mode. 

The pixel shift photography operation of the digital 



camera according to the fourth embodiment will now be 
described with reference to Fig. 26. Fig. 26 shows a 
flowchart for explaining the pixel shift photography 
operation . 

5 In Fig. 26, the camera operation section 8 judges 

whether an instruction to start the normal pixel shift 
photography mode is input or not (step S51) . If an 
instruction to start the normal pixel shift photography mode 
is input, control proceeds to step S52, to thereby execute 

10 the normal pixel shift photography mode . On the other hand, 
in step S51. If an instruction to start the normal pixel 
shift photography mode is not input, control proceeds to 
step S59. In step S59, it is judged whether an instruction 
to start a continuous pixel shift photography is input or 

15 not. If an instruction to start the continuous pixel shift 
photography is input, control proceeds to step S60, to 
thereby execute the continuous pixel shift photography mode . 
On the other hand, in step S59, if an instruction to start 
the continuous pixel shift photography is not input, control 

20 returns to step S51. 

The operation of the normal pixel shift photography 
mode in step S52 to step S58 will now be described. The 
photographing operation of this normal pixel shift 
photography mode is similar to that of the pixel shift 

25 photography in the first embodiment. 



At first, the first photographing is performed and 
the image data is stored in the memory group 32 (step S52) . 
Then, the system controller 30 outputs an adjustment value 
for shifting the image sensing device by one pixel in the 
5 longitudinal direction to the pixel shift drive circuit 6, 
to thereby drive the pixel shift unit 20 via the pixel shift 
drive circuit 6, so that the image sensing device is shifted 
(pixel shift) by one pixel in the longitudinal direction 
(step S53) . The second photographing is performed, with 

10 the image sensing device shifted by one pixel in the 
longitudinal direction, and the image data is stored in the 
memory group 32 (step S54) . The image processing section 
33 generates the image data having a high resolution, by 
combining the first image data and the second image data 

15 stored in the memory group 32 (step S55) . After completion 
of the pixel shift photography, the pixel shift evaluation 
value and reliability calculating section 34 calculates the 
pixel shift evaluation value and the reliability data of 
the image data, for each operation area in the image frame 

20 (see Fig. 12) , based on the first image data and the second 
image data (image data before and after the pixel shift) 
stored in the memory group 32 (step S56), and output the 
pixel shift evaluation value and the reliability data to 
the system controller 30. The system controller 30 judges 

25 whether the pixel shift photography has been performed 



normally, based on the pixel shift evaluation value and the 
reliability data of the image data calculated by the pixel 
shift evaluation value and reliability calculating section 
34 (step S57). Thereafter, the system controller 30 
5 displays the judgment result of the pixel shift photography 
on the display screen in the camera display section 7 (step 
S58) . 

The operation of the continuous pixel shift 
photography mode instepS60tostepS74will now be described . 

10 At first, the system controller 30 sets the value of a 
pre-pixel shift photography frequency counter N to "0" (step 
S60). Then, the first photographing is performed and the 
image data is stored in the memory group 32. The system 
controller 30 then outputs an adjustment value for shifting 

15 the image sensing device by one pixel in the longitudinal 
direction to the pixel shift drive circuit 6, to drive the 
pixel shift unit 20 via the pixel shift drive circuit 6 to 
thereby shift the image sensing device by one pixel in the 
longitudinal direction (pixel shift) (stepS61). Then, the 

20 second photographing is performed, with the image sensing 
device shifted by one pixel in the longitudinal direction, 
and the image data is stored in the memory group 32 (step 
S62) . 

The pixel shift evaluation value and reliability 
25 calculating section 34 calculates the pixel shift evaluation 



value and the reliability data of the image data, for each 
operation area in the image frame (see Fig. 12), based on 
the first image data and the second image data (image data 
before and after the pixel shift) stored in the memory group 
5 32 (step S63) , and output the pixel shift evaluation value 
and the reliability data to the system controller 30 , 

The system controller 30 increments the value of the 
pre-pixel shift photography frequency counter N to "1" (step 
S64). Then, the system controller 30 judges whether the 

10 pixel shift evaluation value is within a predetermined range 
or not (step S65) . As a result of this judgment, if the 
pixel shift evaluation value is without the predetermined 
range, control proceeds to step S66. On the other hand, 
if the pixel shift evaluation value is within the 

15 predetermined range, control proceeds to step S68. 

In step S66, the system controller 30 calculates the 
adjustment value (step S64) based on the pixel shift 
evaluation value and the reliability data of the image data 
calculated by the pixel shift evaluation value and 

20 reliability calculating section 34, and changes the 
adjustment value stored in the memory group 32 to the new 
adjustment value, and control proceeds to step S67. 

In step S67, the system controller 30 judges if the 
value of the pre-pixel shift photography frequency counter 

25 N exceeds the predetermined number of times A (for example. 



the predetermined number of times A = 4 to 5) or not. As 
a result of this judgment, if the value of the pre-pixel 
shift photography frequency counter N is smaller than the 
predetermined number of times A, control returns to step 

,5 S61, to execute the pre-pixel shift photography (step S61 
to step S65) . At this time, as the adjustment value set 
in the pixel shift drive circuit 6, the value calculated 
in step S64 is used. On the other hand, in step S67, if 
the value of the pre-pixel shift photography frequency 

10 counter N exceeds the predetermined number of times A, 
control proceeds to step S68. 

In this manner, the pre-pixel shift photography is 
completed, when the pixel shift evaluation value becomes 
within the predetermined range, or when the pre-pixel shift 

15 photography has been performed for the predetermined number 
of times A. 

In the processing after step S67, the main pixel shift 
photography is performed. A first photographing is first 
performed to store the image data in the memory group 32 

20 (step S68) . The system controller 30 then sets the newest 
adjustment value calculated in the above pre-pixel shift 
photography in the pixel shift drive circuit 6, to drive 
the pixel shift unit 20 via the pixel shift drive circuit 
6, to thereby shift the image sensing device by one pixel 

25 in the longitudinal direction (pixel shift) (step S69). 



Then, the second photographing is performed, with the 
image sensing device shifted by one pixel in the longitudinal 
direction, and the image data is stored in the memory group 
32 (step S70). The image processing section 33 generates 
5 the image data having a high resolution, by combining the 
first image data and the second image data stored in the 
memory group 32 (step S71) . After completion of the pixel 
shift photography, the pixel shift evaluation value and 
reliability calculating section 34 calculates the pixel 

10 shift evaluation value and the reliability data of the image 
data, based on the first image data and the second image 
data (image data before and after the pixel shift) stored 
in the memory group 32 (step S72) , and output the pixel shift 
evaluation value and the reliability data to the system 

15 controller 30 . The system controller 30 j udges whether the 
pixel shift photography has been performed normally, based 
on the pixel shift evaluation value and the reliability data 
of the image data calculated by the pixel shift evaluation 
value and reliability calculating section 34 (step S73) . 

20 Thereafter, the system controller 30 displays the judgment 
result of the pixel shift photography on the display screen 
in the camera display section 7 (step S74) . 

In the above flowchart, the method of calculating the 
pixel shift evaluation value and the reliability data, and 

25 the method of calculating the optimum adjustment value based 



on these are the same as those in the second embodiment, 
and hence the description thereof is omitted. 

In the above flowchart, after the adjustment value 
is set by the pre-pixel shift photography, the main pixel 
5 shift photography is performed again, but a method in which 
photographing is simply completed at the stage where the 
pixel shift evaluation value becomes OK and the last image 
is recorded may be adopted. 

Moreover, though not shown in the above-mentioned 
^r}^ 10 flowchart, the system cent roller 3 0 may j udge that the pixel 
JJ shift unit 20 has a failure, when the calculated adjustment 

« value has exceeded the predetermined number of times or the 

pi predetermined range, in the pre-pixel shift photography in 

the continuous pixel shift photography mode, and display 
La 15 that the pixel shift unit 20 has a failure in the camera 
O display section to thereby inform the photographer of this 

situation, and stop the execution operation of the continuous 
pixel shift photography mode. 

In the continuous pixel shift photography mode, a 
20 combined image and the above-described amount of shift are 
generated for each pixel shift photography in the pre-pixel 
shift photography and the main pixel shift photography, and 
the system controller 30 may store in a storage medium (not 
shown) only a combined image having the smallest amount of 
25 shift, of these combined images. 



In the continuous pixel shift photography mode, a 
plurality of pixel shift photography is performed, while 
increasing or decreasing the adjustment value stored in the 
memory group 32 by a predetermined amount, to calculate the 
5 combined image and the above-described amount of shift for 
each pixel shift photography, and the system controller 30 
may store in a storage medium (not shown) only a combined 
image having the smallest amount of shift, of these combined 
images . 

10 The optimum adjustment value calculated in the 

continuous pixel shift mode is stored in the memory group 
32, and this optimum adjustment value is used also in the 
normal pixel shift mode. Hence, also in the normal pixel 
shift photography, it becomes possible to shift the image 

15 sensing device with the optimum adjustment value. 

As a result, for example, in the case where pixel shift 
photography is performed frequently under the same 
conditions , the continuous pixel shift mode is first executed; 
to set the optimum adjustment value, and at the second time 

20 or after, the normal pixel shift mode can be selected to 
thereby reduce the photographing time. 

As described above, according to the fourth embodiment, 
when the continuous pixel shift mode is selected, the 
pre-pixel shift photography is first executed, to calculate 

25 a new adjustment value based on the calculated pixel shift 



evaluation value. Then the pre-pixel shift photography is 
performed again with the calculated new adjustment value. 
This operation is repetitively executed, and the main pixel 
shift photography is executed based on the final adjustment 
5 value. As a result, the image sensing device can be shifted 
with the optimum adjustment value, and the image sefrsi.ng 
unit can be displaced by a desired amount, thereby high 
quality image can be obtained. 

According to the fourth embodiment, the amount of shift 
10 is calculated by the pre-pixel shift photography in the 
continuous pixel shif tmode, and the optimum adj us tment value 
is calculated based on the calculated amount of shift. As 
a result, an accurate and optimum adjustment value can be 
calculated . 

15 According to the fourth embodiment, the pixel shift 

evaluation value and reliability calculating section 34 
calculates the pixel shift evaluation value (the amount of 
shift or the coincidence degree) for each operation area 
in the image frame. As a result, it can be detected whether 

20 the pixel shift evaluation value (the amount of shift or 
the coincidence degree) is adequate or not in a plurality 
of points in the image, and a precise adjustment value can 
be obtained with regard to the shift degree of the whole 
image . 

25 According to the fourth embodiment, the pixel shift 



evaluation value and reliability calculating section 34 
calculates the pixel shift evaluation value (the amount of 
shift or the coincidence degree) for each operation area 
in the image frame. As a result, it can be detected whether 
5 the pixel shift evaluation value (the amount of shift or 
the coincidence degree) is adequate or not in a plurality 
of points in the image, and a precise adjustment value can 
be obtained with regard to the shift degree of the whole 
image . 

10 According to the fourth embodiment, the reliability 

data is calculated based on the contrast information of a 
subject (image) , and the pixel shift evaluation value in 
the operation area of an image having a low contrast and 
hence having a large probability of an error can be taken 

15 into account, thereby enabling highly accurate calculation 
of the adjustment value. 

According to the fourth embodiment, the system 
controller 30, using the pixel shift photography control 
unit executes the above-described pre-pixel shift 

20 photography repetitively for a predetermined number of times, 
or until the pixel shift evaluation value becomes within 
a predetermined range. Hence, an image having a sufficient 
effect of pixel shift can be generally obtained, and such 
a situation that a camera does not stop can be avoided. 

25 According to the fourth embodiment, the system 



controller 30 judges that the pixel shift mechanism has a 
failure, when a calculated new adjustment value exceeds a 
predetermined number of times or a predetermined range, in 
the pre-pixel shift photography, displays this matter in 
5 the camera display section 7 for informing the photographer. 
As a result, pixel shift photography is not performed in 
an inadequate condition, and the photographer can be prompted 
to make a service call. 

According to the fourth embodiment, in the pre-pixel 

10 shift photography and the main pre-pixel shift photography, 
only a combined image having the smallest amount is stored 
in the recording medium. As a result, the memory capacity 
of the recording medium can be reduced, and the best image 
can be always obtained. 

15 The above described image input method may be such 

that a program prepared in advance is executed by a personal 
computer or a computer such as a work station. This program 
is executed by being read from a recording medium that can 
be read by a computer such as hard disk, floppy disk, CD-ROM, 

20 MO, and DVD. This program can be also distributed via the 
above recording medium, or via a network such as Internet, 
as a transmission medium. 

As described above, according to the invention of one 
aspect, the photographic optical system forms an image of 

25 the subject in the predetermined position. The image 



sensing unit images the subj act image and outputs image data . 
The pixel shift mechanism control unit controls the pixel 
shift mechanism to displace the image sensing unit by a 
predetermined amount. The image combined unit generates 
5 image data for one image by combining image data for the 
plurality of images output before and after the displacement 
of the image sensing unit . The judgment unit judges whether 
the pixel shift photography has been normally performed or 
not, based on the image data for a plurality of images output 

10 before and after the displacement of the image sensing unit. 
As a result, a failure of pixel shift photography caused 
by movement of hands, movement of the subject, or a failure 
of the pixel shift mechanism can be informed to a photographer, 
thereby a pixel shifted image desired by the photographer 

15 can be obtained by re-photographing or the like. 

Moreover , the pixel shift evaluation value calculation 
unit calculates a pixel shift evaluation value for judging 
whether the pixel shift photography has been normally 
performed or not, based on the image data for a plurality 

20 of images output before and after the displacement of the 
image sensing unit, and the pixel shift photography judgment 
unit judges whether the pixel shift photography has been 
normally performed or not, based on the pixel shift 
evaluation value calculated by the pixel shift evaluation 

25 value calculation unit. As a result, in addition to the 



effect according to the first aspect, it becomes possible 
to judge if the pixel shift photography is successful or 
not with a simple method. 

Furthermore, the pixel shift evaluation value 
5 calculation unit calculates an amount of shift between the 
image data for the plurality of images output before and 
after the displacement of the image sensing unit by the 
predetermined amount, as the pixel shift evaluation value, 
and the pixel shift photography judgment unit judges whether 

10 the pixel shift photography has been performed normally, 
based on the amount of shift calculated by the pixel shift 
evaluation value calculation unit . As a result, in addition 
to the effect according to the second aspect, it becomes 
possible to judge if the pixel shift photography is 

15 successful or not more accurately. 

Moreover, the pixel shift evaluation value calculation 
unit calculates each coincidence degree for a plurality of 
areas of the image data, when calculating the coincidence 
degree between the image data for the plurality of images. 

20 As a result, in addition to the effect of the invention 
according to the third aspect, it becomes possible to detect 
if the coincidence degree is adequate or not in a plurality 
of points in the image, thereby enabling accurate judgment 
about the shift degree of the whole image. 

25 Furthermore, the pixel shi ft photography j udgment unit 



judges that the pixel shift photography has been performed 
normally when a part of or the whole of the coincidence degree 
in the plurality of areas calculated by the pixel shift 
evaluation value calculation unit is within a predetermined 
range. As a result, in addition to the effect of the 
invention according to the fourth aspect, it becomes possible 
to judge unsuccessful pixel shift photography more 
accurately . 

Moreover, the pixel shift photography judgment unit 
judges that the pixel shift photography has not been 
performed normally when a part of or the whole of the 
coincidence degree in the plurality of areas calculated by 
the pixel shift evaluation value calculation unit is out 
of the predetermined range, and there is a predetermined 
relation in the coincidence degree in the plurality of areas . 
As a result, in addition to the effect of the invention 
according to the fourth aspect, it becomes possible to judge 
a failure of the pixel shift photography more accurately. 

Furthermore, thepixel shi ft photography j udgment unit 
judges that the pixel shift photography has been performed 
normally, but a part of the subject has moved, in the case 
where a part of or the whole of the coincidence degree in 
the plurality of areas calculated by the pixel shift 
evaluation value calculation unit is out of the predetermined 
range, but there is no predetermined relation in the 



coincidence degree in the plurality of areas. As a result, 
in addition to the effect of the invention according to the 
fourth aspect, it becomes possible to judge a movement of 
the subject. 

5 Moreover, the reliability evaluation unit calculates 

reliability data indicating the reliability of each 
coincidence degree in the plurality of areas calculated by 
the pixel shift evaluation value calculation unit , and judges 
whether the pixel shift photography has been performed 

10 normally, based on the coincidence degree in the plurality 
of areas and the reliability data. As a result, in addition 
to the effect according to either one of the fourth to the 
seventh aspects, at the time of judgment of the pixel shift 
photography, the coincidence degree of an operation area 

15 having a large influence of noisemay not be used, orweighting 
therefor can be made small, thereby enabling prevention of 
erroneous judgment of pixel shift photography. 

Furthermore, the pixel shift evaluation value 
calculation unit calculates the coincidence degree between 

20 the target image data when the image data output by the image 
sensing unit before being displa,ced by the predetermined 
amount is shifted by the predetermined amount, and the image 
data output after the displacement of the image sensing unit 
by the predetermined amount, as the pixel shift evaluation 

25 value, and the pixel shift photography judgment unit judges 



whether the pixel shift photography has been performed 
normally, based on the coincidence degree calculated by the 
pixel shift evaluation value calculation unit . As a result, 
in addition to the effect according to the second aspect, 
it becomes possible to judge whether the pixel shift 
photography is successful or not more accurately. 

Moreover, thepixel shift evaluation value calculation 
unit calculates the coincidence degree, respectively, for 
the plurality of areas of the image data, at the time of 
calculating the coincidence degree. As a result, in 
addition to the effect of the invention according to the 
ninth aspect, it becomes possible to detect whether the 
coincidence degree is adequate or not in a plurality of points 
in the image, thereby enabling accurate judgment about the 
coincidence degree of the whole image. 

Furthermore, the pixel shi ft photography j udgraent unit 
judges that the pixel shift photography has been performed 
normally when a part of or the whole of the coincidence degree 
in the plurality of areas calculated by the pixel shift 
evaluation value calculation unit is within a predetermined 
range. As a result, in addition to the effect of the 
invention according to the tenth aspect, it becomes possible 
to judge a failure of the pixel shift photography more 
accurately. 

Moreover, the pixel shift photography judgment unit 
90 



judges that the pixel shift photography has not been 
performed normally when a part of or the whole of the 
coincidence degree of the plurality of areas calculated by 
the pixel shift evaluation value calculation unit is out 
of the predetermined range, and there is a predetermined 
relation in the coincidence degree in the plurality of areas . 
As a result, in addition to the effect of the invention 
according to the tenth aspect, it becomes possible to judge 
a failure of the pixel shift photography more accurately. 

Furthermore , the pixel shift photography j udgment unit 
judges that the pixel shift photography has been performed 
normally, but a part of the subject has moved, in the case 
where a part of or the whole of the coincidence degree in 
the plurality of areas calculated by the pixel shift 
evaluation value calculation unit is out of the predetermined 
range, but there is no predetermined relation in the 
coincidence degree in the plurality of areas. As a result, 
in addition to the effect of the invention according to the 
tenth aspect, it becomes possible to judge a movement of 
the subject. 

Moreover, the reliability evaluation unit calculates 
reliability data indicating the reliability of each 
coincidence degree in the plurality of areas calculated by 
the pixel shift evaluation value calculation unit, and the 
pixel shift photography judgment unit judges whether the 

91 



pixel shift photography has been performed normally, based 
on the coincidence degree in the plurality of areas and the 
reliability data. As a result, in addition to the effect 
of the invention according to either one of the tenth aspect 
to the thirteenth aspect, at the time of judgment of the 
pixel shift photography, the coincidence degree of an 
operation area having a large influence of noise may not 
be used, or weighting therefor can be made small, thereby 
enabling prevention of erroneous judgment of pixel shift 
photography . 

Furthermore, the informing unit informs of the 
judgment result of the pixel shift photography j udgment unit . 
As a result, in addition to the effects of the inventions 
according to the first aspect to the fourteenth aspect, it 
can be informed to the photographer that the pixel shift 
photography has been abnormally performed, so that the 
photographer can take suitable measures (measures for 
preventing movement of hands or restricting movement of the 
subject, making a service call, etc.). 

According to the invention of another aspect, the 
photographic optical system forms an image of a subject in 
a predetermined position, the image sensing unit images the 
subject image and outputs image data, the pixel shift 
mechanism control unit controls the pixel shift mechanism 
to displace the image sensing unit by a predetermined amount . 



The image combining unit generates image data for one image 
by combining image data for the plurality of images output 
before and after the displacement of the image sensing unit. 
The pixel shift evaluation value calculation unit calculates 
a pixel shift evaluation value for judging whether the pixel 
shift photography has been normally performed or not, based 
on the image data for a plurality of images output before 
and after the displacement of the image sensing unit. The 
storage control unit stores the pixel shift evaluation value 
calculated by the pixel shift evaluation value calculation 
unit in a storage unit, and the failure judgment unit judges 
a failure of the pixel shift mechanism, based on the pixel 
shift evaluation values obtained by performing the pixel 
shift photography for several times and stored in the storage 
unit. As a result, when an abnormal situation occurs in 
the pixel shift photography, it can be judged whether it 
is temporary or permanent (= failure), thereby enabling 
judgment of a failure of the pixel shift mechanism. 

Moreover, the pixel shift evaluation value calculation 
unit calculates a coincidence degree between the image data 
for the plurality of images output before and after the 
displacement of the image sensing unit by the predetermined 
amount, as the pixel shift evaluation value. As a result, 
in addition to the effect of the invention according to the 
sixteenth aspect, it becomes possible to judge a failure 



of the pixel shift mechanism accurately. 

Furthermore, the pixel shift evaluation value 
calculation unit calculates each coincidence degree for a 
plurality of areas of the image data, when calculating the 
coincidence degree between the image data for the plurality 
of images. As a result, in addition to the effect of the 
invention according to the seventeenth aspect, it becomes 
possible to detect if the coincidence degree is adequate 
or not in a plurality of points in the image, thereby enabling 
accurate judgment about the shift degree of the whole image. 

Moreover, the reliability evaluation unit calculates 
reliability data indicating the reliability of the 
coincidence degree in the plurality of areas calculated by 
the pixel shift evaluation value calculation unit, the 
storage control unit stores the reliability data calculated 
by the reliability evaluation unit in the storage unit, 
corresponding to the data of coincidence degree, and the 
failure judgment unit judges the existence of abnormality 
in the pixel shift mechanism, based on the pixel shift 
evaluation values obtained by performing the pixel shift 
photographing for several times stored in the storage unit 
and the reliability data. As a result, in addition to the 
effect of the invention according to the eighteenth aspect, 
it becomes possible to minimize noise at the time of judging 
a failure of the pixel shift mechanism, thereby enabling 



judgment of a failure in the pixel shift mechanism 
accurately . 

Furthermore, the reliability evaluation unit 
calculates reliability data indicating the reliability of 
the coincidence degree in the plurality of areas calculated 
by the pixel shift evaluation value calculation unit, the 
storage control unit corrects the coincidence degree based 
on the reliability data calculated by the reliability 
evaluation unit and stores the corrected coincidence degree 
in the storage unit, and the failure judgment unit judges 
the existence of abnormality in the pixel shift mechanism, 
based on the corrected coincidence degree of the last several 
times stored in the storage unit. As a result, in addition 
to the effect of the invention according to the nineteenth 
aspect, it becomes possible to reduce the data amount stored 
in the storage unit. 

Moreover, the pixel shift evaluation value calculation 
unit calculates the coincidence degree between the target 
image data when the image data output by the image sensing 
unit before being displaced by a predetermined amount is 
shifted by the predetermined amount, and the image data 
output after the displacement of the image sensing unit by 
the predetermined amount, as the pixel shift evaluation value . 
As a result, in addition to the effect of the invention 
according to the sixteenth aspect, it becomes possible to 



judge a failure in the pixel shift mechanism accurately. 

Furthermore, the pixel shift evaluation value 
calculation unit calculates the coincidence degree, 
respectively, for the plurality of areas of the image data, 
at the time of calculating the coincidence degree. As a 
result, in addition to the effect of the invention according 
to the twenty-first aspect, it becomes possible to detect 
if the coincidence degree is adequate or not in a plurality 
of points in the image, thereby enabling accurate judgment 
about the coincidence degree of the whole image. 

Moreover, the reliability evaluation unit calculates 
reliability data indicating the reliability of the 
coincidence degree in the plurality of areas calculated by 
the pixel shift evaluation value calculation unit, the 
storage control unit stores the reliability data calculated 
by the reliability evaluation unit in the storage unit, 
corresponding to the data of coincidence degree, and the 
failure judgment unit judges the existence of abnormality 
in the pixel shift mechanism, based on the pixel shift 
evaluation values of photographing of the last several times 
stored in the storage unit and the reliability data. As 
a result, in addition to the effect of the invention according 
to the twenty-second aspect, it becomes possible to minimize 
noise at the time of judging a failure in the pixel shift 
mechanism, thereby enabling judgment of a failure in the 



pixel shift mechanism accurately. 

Furthermore, the reliability evaluation unit 
calculates reliability data indicating the reliability of 
the coincidence degree in the plurality of areas calculated 
by the pixel shift evaluation value calculation unit, the 
storage control unit corrects the coincidence degree based 
on the reliability data calculated by the reliability 
evaluation unit and stores the corrected coincidence degree 
in the storage unit, and the failure judgment unit judges 
the existence of abnormality in the pixel shift mechanism, 
based on the corrected coincidence degree of the last several 
times stored in the storage unit. As a result, in addition 
to the effect of the invention according to the twenty-second 
aspect, it becomes possible to reduce the data amount stored 
in the storage unit. 

Moreover, the pixel shift mechanism failure informing 
unit informs of a failure in the pixel shift mechanism, when 
it is judged by the failure judgment unit that the pixel 
shift mechanism has a failure. As a result, in addition 
to the effect of the invention according to either one of 
the sixteenth to twenty-fourth aspects, the photographer 
can take suitable measures (suspension of use of the pixel 
shift mode, making a service call, etc.). 

Furthermore, the self calibration unit judges whether 
it is necessary or not to change the adjustment value for 



driving the pixel shift mechanism, and if it is necessary 
to change the adjustment value, changes the adjustment value 
stored in the adjustment value storage unit. As a result, 
in addition to the effect of the invention according to either 
one of the sixteenth aspect to the twenty-fifth aspect, at 
the time of a negligible failure of the pixel shift mechanism, 
the adjustment value for driving the pixel shift mechanism 
can be automatically changed, to thereby restore the normal 
condition . 

Moreover, the input unit instructs to change a new 
adjustment value stored in the adjustment value storage unit 
to a value before the change or to an initial value by the 
pixel shift self calibration unit . As a result, in addition 
to the effect of the invention according to the twenty-sixth 
aspects, suitable pixel shift photography can be performed 
depending on the use environment. 

Furthermore, the frequency input unit sets the 
photographing frequency for the historical pixel shift 
evaluation values stored in the storage unit, and the self 
calibration unit calculates a new adjustment value, based 
on the historical pixel shift evaluation values for the 
number of times of photographing set by the frequency input 
unit. As a result, in addition to the effect of the invention 
according to the twenty-sixth aspects, the adjustment time 
desired by the photographer can be set. 



According to the invention of still another aspect, 
the photographic optical system forms an image of the sub j ect 
in the predetermined position, the image sensing unit images 
the subject image and outputs image data, and the pixel shift 
mechanism control unit controls the pixel shift mechanism 
based on the adjustment value stored in the adjustment value 
storage unit to displace the image sensing unit by a 
predetermined amount. The image combining unit generates 
image data for one image by combining image data for the 
plurality of images output before and after the displacement 
of the image sensing unit . The pixel shift evaluation value 
calculation unit judges whether the pixel shift photography 
has been normally performed or not, based on the image data 
for a plurality of images output before and after the 
displacement of the image sensing unit. The selection unit 
selects the pixel shift calibration mode. When the pixel 
shift calibration mode is selected, the self calibration 
unit executes pre-pixel shift photography once or plural 
times, re-calculates the adjustment value based on the pixel 
shift evaluation value for each pixel shift photography, 
calculated by the pixel shift evaluation value calculation 
unit, to thereby calculate a new adjustment value, and 
changes the adjustment value stored in the adjustment value 
storage unit to the new adjustment value. As a result, the 
photographer can perform calibration for the adjustment 



value, anytime at his discretion, thereby the best pixel 
shifted image can be always obtained. 

Moreover, the pixel shift evaluation value calculation 
unit calculates a coincidence degree between the image data 
for the plurality of images output before and after the 
displacement of the image sensing unit by the predetermined 
amount, as the pixel shift evaluation value, and the self 
calibration unit calculates the new adjustment value based 
oh the coincidence degree. As a result, in addition to the 
effect of the invention according to the twenty-ninth aspect, 
it becomes possible to calculate an accurate adjustment 
value . 

Furthermore, when the pixel shift calibration mode 
is selected, the self calibration unit sets an adjustment 
value larger than the adjustment value stored in the 
adjustment value storage unit by a predetermined amount and 
an adjustment value smaller than that by a predetermined 
amount in the pixel shift mechanism, and performs pixel shift 
photography twice, to re-calculate the adjustment value 
based on the coincidence degree between the two pixel shift 
photography calculated by the pixel shift evaluation value 
calculation unit to thereby calculate a new ad j us tment value . 
As a result, in addition to the effect of the invention 
according to the thirtieth aspect, a relatively accurate 
adjustment value can be calculated with a fewer number of 



times of photographing. 

Moreover, when the pixel shift calibration mode is 
selected, the self calibration unit sets the adj us tment value 
stored in the adjustment value storage unit in the pixel 
shift mechanism to perform one pixel shift photography, and 
refers to the relational data between the adjustment value 
registered in advance and the coincidence degree, based on 
the coincidence degree calculated by the pixel shift 
evaluation value calculation unit, to thereby calculate the 
new adjustment value . As a result, in addition to the effect 
of the invention according to the thirtieth aspect, the 
adjustment value can be calculated within a short period 
of time. 

Furthermore, the pixel shift evaluation value 
calculation unit calculates, as the pixel shift evaluation 
value, the coincidence degree between a target image data 
when the image data output by the image sensing unit before 
being displaced by a predetermined amount is shifted by the 
predetermined amount, and the image data output after the 
displacement of the image sensing unit by the predetermined 
amount, and the self calibration unit calculates the new 
adjustment value based on the coincidence degree. As a 
result, in addition to the effect of the invention according 
to the twenty-ninth aspect, the adjustment value can be 
accurately calculated. 



Moreover, when calculating the amount of shift or the 
coincidence degree, the pixel shift evaluation value 
calculation unit calculates the amount of shift or the 
coincidence degree, respectively, for the plurality of areas 
of the image data. As a result, in addition to the effect 
of the invention according to either one of the twenty-ninth 
to the thirty-third aspects, it becomes possible to detect 
whether the pixel shift evaluation value (the amount of shift 
or the coincidence degree ) is adequate or not in the plurality 
of points in the image, thereby more precise adjustment value 
can be obtained about the shift degree (coincidence degree) 
of the whole image. 

Furthermore, the reliability evaluation unit 
calculates reliability data indicating the reliability of 
each coincidence degree or each coincidence degree in the 
plurality of areas calculated by the pixel shift evaluation 
value calculation unit, and the self calibration unit 
calculates the new adjustment value based on the amount of 
shift or the coincidence degree, and the reliability data. 
As a result, in addition to the effect of the invention 
according to the thirty-fourth aspect, it becomes possible 
to minimize the influence of noise of the image at the time 
of calculating a new adjustment value, thereby enabling 
accurate calculation of the adjustment value. 

Moreover, the an unacceptable subject informing unit 
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which informs that it is necessary to change the subject 
or to change the distance of subject, when a part of or the 
whole of the reliability data calculated by the reliability 
evaluation unit does not reach a predetermined level. As 
a result, in addition to the effect of the invention according 
to the thirty-fifth aspect, it becomes possible to prevent 
inadequate calibration from being performed, thereby 
enabling prevention of a failure in the pixel shift 
photography . 

Furthermore, the pixel shift mechanism failure 
judgment unit judges whether the new adjustment value 
calculated by the self calibration unit is within a 
predetermined range or not, and when it is without the 
predetermined range, judges that the pixel shift mechanism 
has a failure, and the pixel shift mechanism failure 
informing unit informs of a failure of the pixel shift 
mechanism, when it is judged that the pixel shift mechanism 
has a failure by the pixel shift mechanism failure judgment 
unit. As a result, in addition to the e f feet of the invention 
according to the thirty-sixth aspect, it can be prevented 
toperforrapixel shi ft photography in an inadequate condition, 
thereby the photographer can take adequate measures 
(suspension of use of the pixel shift mode, making a service 
call, etc . ) . 

According to the invention of still another aspect. 



the photographic optical system forms an image of the subject 
in the predetermined position, the image sensing unit images 
the subject image and outputs image data, and the pixel shift 
mechanism control unit controls the pixel shift mechanism 
based on the adjustment value stored in the adjustment value 
storage unit to displace the image sensing unit by a 
predetermined amount. The image combining unit generates 
image data for one image by combining image data for the 
plurality of images output before and after the displacement 
of the image sensing unit . The pixel shift evaluation value 
calculation unit calculates a pixel shift evaluation value 
for judging whether the pixel shift photography has been 
normally performed or not, based on the image data for a 
plurality of images output before and after the displacement 
of the image sensing unit. The pixel shift photography 
control unit controls to execute an operation repetitively 
in which, when a continuous pixel shift mode is selected, 
a pre-pixel shift photography is executed by setting an 
adjustment value in the pixel shift mechanism, the pixel 
shift evaluation value calculation unit calculates thepixel 
shift evaluation value to thereby calculate a new adjustment 
value based on the calculated pixel shift evaluation value, 
and pre-pixel shift photography is performed again with the 
calculated new adjustment value. As a result, an optimum 
adjustment value can be calculated, and it becomes possible 



to drive the pixel shift mechanism based on this optimum 
adjustment value, and hence high quality image can be 
obtained . 

Moreover, the pixel shift photography control unit 
determines the final adjustment value based on the adjustment 
value calculated by the repetitively performed pre-pixel 
shift photography, and controls so that the final pixel shift 
photography is executed, basedon the final adj ustment value . 
As a result, the final pixel shift photography can be 
performed with the optimum adjustment value, and hence high 
quality image can be obtained. 

Furthermore, the pixel shift evaluation value 
calculation unit calculates the coincidence degree between 
the image data for the plurality of images output before 
and after the displacement of the image sensing unit by a 
predetermined amount, as the pixel shift evaluation value, 
and the pixel shift photography control unit calculates the 
new adjustment value based on the coincidence degree. As 
a result, in addition to the effect of the invention according 
to the thirty-eighth aspect or the thirty-ninth aspect, it 
becomes possible to calculate the adjustment value more 
accurately . 

Moreover, the pixel shift evaluation value calculation 
unit calculates the coincidence degree between the target 
image data when the image data output by the image sensing 
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unit before being displaced by the predetermined amount is 
shifted by the predetermined amount^ and the image data 
output after the displacement of the image sensing unit by 
a predetermined amount, as the pixel shift evaluation value; 
and the pixel shift photography control unit calculates the 
new adjustment value based on the coincidence degree. As 
a result, in addition to the effect of the invention according 
to the thirty-eighth aspect or the thirty-ninth aspect, it 
becomes possible to calculate the adjustment value more 
accurately. 

Furthermore, the pixel shift evaluation value 
calculation unit calculates the amount of shift or the 
coincidence degree, respectively, for a plurality of areas 
of the image data, when calculating the amount of shift or 
the coincidence degree. As a result, in addition to the 
effect of the invention according to the fortieth aspect 
or the forty-first aspect, it can be detected whether the 
pixel shift evaluation value calculation unit (the amount 
of shift or the coincidence degree) is adequate or not in 
the plurality of points in the image, thereby more precise 
adjustment value can be obtained with regard to the shift 
degree (coincidence degree) of the whole image. 

Moreover, the reliability evaluation unit calculates 
reliability data indicating the reliability of each 
coincidence degree or each coincidence degree in the 



plurality of areas calculated by the pixel shift evaluation 
value calculation unit, and the pixel shift photography 
control unit calculates the new adjustment value, based on 
the amount of shift or the coincidence degree, and the 
reliability data. As a result, in addition to the effect 
of the invention according to the forty-second aspect, it 
becomes possible to minimize the influence of noise of the 
image at the time of calculating a new adjustment value, 
thereby enabling accurate calculation of the adjustment 
value . 

Furthermore, the pixel shift photography control unit 
repetitively executes the pre-pixel shift photography for 
a predetermined number of times, or until the pixel shift 
evaluation value becomes within a predetermined range. As 
a result, in addition to the effect of the invention according 
to the thirty-ninth to the forty-third aspects, a high 
quality image can be obtained, and such a situation that 
the camera does not stop can be avoided. 

Moreover, the pixel shift photography control unit 
judges that the pixel shift mechanism has a failure, in the 
case where the calculated new adjustment value exceeds a 
predetermined number of times or a predetermined range, in 
the pre-pixel shift photography, and informs of this matter . 
As a result, in addition to the effect of the invention 
according to the thirty-ninth to the forty-fourth aspects. 



pixel shift photography is not performed in an inadequate 
condition, thereby it becomes possible to urge the 
photographer to make a service call. 

Furthermore, the pixel shift photography control unit 
stores, in the image storage unit, only the combined image 
having the best pixel shift evaluation value calculated by 
the pixel shift evaluation value calculation unit in the 
pre-pixel shift photography and the main pixel shift 
photography. As a result, in addition to the effect of the 
invention according to either one of the thirty-ninth to 
the forty-fourth aspects, the memory capacity of the 
recording medium can be reduced, thereby the best image can 
be always obtained. 

Moreover, the mode selection unit selects the normal 
pixel shift photography mode for generating the combined 
image by one pixel shift photography and the continuous pixel 
shift photography mode. As a result, it becomes possible 
to select a plurality of pixel shift photography modes 
depending on the preference of the photographer. 

Furthermore, the reliability evaluation unit 
calculates the reliability data based on the contrast of 
the image within the range of each operation area. As a 
result, in addition to the effect of the invention according 
to either one of the above-mentioned aspects, the pixel shift 
evaluation value in the operation area of an image having 
108 



a low contrast and hence having a large probability of error 
can be taken into consideration, and hence a more accurate 
adjustment value can be calculated. 

According to the invention of still another aspect, 
a subject image is imaged on the image sensing unit to output 
first image data, the image sensing unit is displaced by 
a predetermined amount, the subject image is imaged on the 
image sensing unit displaced by the predetermined amount 
to output second image data, image data for one image is 
generated by combining the first image data and the second 
image data, and it is judged whether the pixel shift 
photography has been normally performed or not, based on 
the first image data and the second image data. As a result, 
a failure of the pixel shift photography caused by the 
movement of hands, movement of the subject, a problem in 
the mechanism, or the like can be informed to the photographer, 
and hence the photographer can obtain a desired pixel shifted 
image by re-photographing or the like. 

According to the invention of still another aspect, 
a subject image is imaged on the image sensing unit to output 
first image data, the image sensing unit is displaced by 
a predetermined amount by the pixel shift mechanism, the 
subject image is imaged on the image sensing unit displaced 
by the predetermined amount to output second image data, 
and image data for one image is generated by combining the 
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first image data and the second image data. The pixel shift 
evaluation value for judging whether the pixel shift 
photography has been normally performed or not is calculated 
based on the first image data and the second image data. 
The pixel shift evaluation value is stored in the storage 
unit^ and it is judged if there is a failure of the pixel 
shift mechanism, based on the pixel shift evaluation values 
obtained by performing the pixel shift photography for 
several times and stored in said storage unit and stored 
in the storage unit . As a result, when an abnormal situation 
occurs in the pixel shift photography, it can be judged 
whether it is temporary or permanent (= failure) , thereby 
enabling judgment of a failure of the pixel shift mechanism. 

According to the invention of still another aspect, 
a subject image is imaged on the image sensing unit to output 
first image data, the pixel shift mechanism is driven based 
on the adjustment value stored in the storage unit to displace 
the image sensing unit by a predetermined amount, and the 
subject image is imaged on the image sensing unit displaced 
by the predetermined amount to output second image data. 
One image data is generated based on the first image data 
and the second image data, and when the self calibration 
mode is selected, pre-pixel shift photography is executed 
once or plural times, to calculate a pixel shift evaluation 
value for judging whether the pixel shift photography has 



been normally performed or not based on the first image data 
and the second image data. Based on the calculated pixel 
shift evaluation value, a new adjustment value is calculated 
by re-calculating the adjustment value, to change the 
5 adjustment value stored in the storage unit to the new 
adjustment value . As a result, the photographer can per form 
calibration of the adjustment value anytime at his discretion, 
thereby the best pixel shifted image can be always obtained. 
According to the invention of still another aspect, 

10 a subject image is imaged on the image sensing unit to output 
first image data, the pixel shift mechanism is driven based 
on the adj ustment value stored in the storage unit to displace 
the image sensing unit by a predetermined amount, and the 
subject image is imaged on the image sensing unit displaced 

15 by the predetermined amount to output second image data. 
The pixel shift evaluation value for judging whether the 
pixel shift photography has been normally performed or not 
is calculated based on the first image data and the second 
image data. A new adjustment value is calculated by 

20 re-calculating the adjustment value based on the calculated 
pixel shift evaluation value, to change the adjustment value 
stored in the storage unit to the new adjustment value. The 
above operation is repeated for a predetermined number of 
times, or until the new adjustment value becomes within a 

25 predetermined range, and pixel shift photography is 



performed with the final adjustment value. As a result, 
the pixel shift mechanism can be driven with the optimum 
adjustment value, and hence high quality image can be 
obtained . 

5 According to the invention of still another aspect, 

each step of the method according to this invention is 
realized byexecuting the computer program, that is stored 
in a recording medium, by the computer. As a result, even 
if movement of hands, movement of the subject, performance 

10 change of the pixel shift mechanism or the like occurs, it 
becomes possible to prevent unsuccessful pixel shift 
photography as much as possible. 

The present document incorporates by reference the 
entire contents of Japanese priority document, 2001-012498 

15 filed in Japan on January 19, 2001. 

Although the invention has been described with respect 
to a specific embodiment for a complete and clear disclosure, 
the appended claims are not to be thus limited but are to 
be construed as embodying all modifications and alternative 

20 constructions that may occur to one skilled in the art which 
fairly fall within the basic teaching herein set forth. 
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